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DESIGNING LEARNING AND TECHNOLOGY 



FOR EDUCATIONAL REFORM 

Introduction 

This paper is based in part on an earlier literature review on technology effectiveness 
conducted for the Illinois State Board of Education (see Jones, Valdez, & Rasmussen, 1994). 
The five technologies reviewed were computers, traditional distance education, two-way 
interactive telecommunications, multimedia, and the Internet. A major outcome of the 
earlier research was a framework to assist schools and policymakers in evaluating specific 
technologies and technology-enhanced curricula. This paper both summarizes and extends 
the earlier study. It also adds some tools for technology evaluation as well as a rationale for 
using regions to distribute information resources and services electronically. 

In this review, we examine traditional and emergent definitions of technology 
effectiveness from various strands of research. We identify a major paradigm shift from a 
focus on student achievement defined solely by standardized tests to one of diverse indicators 
of learning and educational reform defined by recent research on learning and Goals 2000. 
This is a shift from traditional learning and course designs to models of learning involving 
more interactivity, more connectivity among schools, more collaboration among teachers and 
students, more involvement of the teacher as facilitator, and more emphasis on the 
technology as a tool for learning, collaboration, curriculum development, and professional 
development. 

We present here an analytic framework using dimensions of technology and learning to 
create an optimum relationship between quality learning as we have come to know it and 
high performance technology. In this paper, we explore this framework and elaborate on it, 
looking at some of the issues people are dealing with in technology and learning and 
identifying how this framework can come into play. 

This paper is organized into parts. Part 1 presents an overview of the framework and 
its potential uses and importance for educators and policymakers. In Part 2, we discuss the 
new consensus on learning, policy, and the technology capabilities needed to support learning 
and reform. Part 3 considers trends in technologies and agencies that focus on networked 
information resources and developing learning communities. In Part 4, we present the 
concept of regionality as a unit of operation for technology; distribution of services and 
resources; and experimentation of new designs in technology, software, and agency. Part 5 
notes some critical next steps for research. Finally, Part 6 explores policy recommendations 
following from the analysis as a whole. 
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1. Overview of the Technology Effectiveness Framework 

The Technology Effectiveness Framework was developed to assist educators, researchers, 
and policymakers in evaluating technology and technology-enhanced programs/curricula 
against specific reform goals for a school, district, state, or service agency. It may also be 
used by schools to guide them as they select and work toward specific curricular goals to 
promote engaged learning. Additionally, researchers, curriculum developers, and staff 
developers could use this framework to design technologies and technology-enhanced 
programs. 

The Analytic Variables for the Framework 

The overarching concept that drives the framework proposed here is that technology 
effectiveness can be defined as the intersection of two continua (see Table I on following 
page). The horizontal axis is learning, which progresses from passive at the low end of the 
continuum to engaged and sustained at the high end. To help conceptualize this continuum, 
we selected a set of indicators for engaged learning and instructional reform, for which there 
is increasing consensus— developed by Means and her colleagues (1994)— and enhanced it by 
adding other indicators. The vertical axis is technology performance, which progresses 
from low to high. We define six indicators of technology performance derived from recent 
research on instructional design and technology-enhanced learning. Two sets of indicators 
are described in detail in Part 2, 

When we cross the two continua, four major learning and technology patterns emerge: 

Category A — Engaged learning and high technology performance 
Category B — Engaged learning and low technology performance 
Category C — Passive learning and high technology performance 
Category D — Passive learning and low technology performance 

The Value of the Framework 

Using this framework, we have identified swrne major directions for policymakers to 
consider regarding resource and infrastructure support to schools, especially when using 
technology as a tool for classroom and school restructuring. Specifically, we noted four 
positive (desirable) trajectories: 

1, Type I trajectory is D — > B, This is movement 
from passive learning and low technology performance 
to engaged learning and low technology performance. 
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TABLE 1 



THE LEARNING AND TECHNOLOGY INTERFACE 



(C) 

Examples 

-Closed integrated learning systems 
focusing on low-level objectives and 
standardized, objective assessments 

-Traditional distance education used to 
transmit information from a central source 
and focused on low-level objectives and 
assessments (talking head) 

-Connections to homes that are linked only 
to closed networks for the school and 
vendor and perhaps to other schools using 
the same vendor 



(A) 



Examples 



-Networked projects with challenging tasks; 
access to Internet; integrated multimedia 
capabilities including CD-ROM, two-way 
video conferencing, access to professionals 

- Distance education networked with 
computers; challenging tasks; linked to 
work with real-world professionals and 
data; two-way video 

-Advanced tools and high-technology 
museum exhibits that are interactive and 
support high-level thinking 



(D) 

Examples 

-Computer-based instruction/drill and 
practice focusing on low-level objectives 

-Instructional television focused on low- 
level objectives 

-Video and audio used to transmit 
information as a lecture or talking head 

-Teaching a computer language or word 
processing as an end in itself as technology 
literacy 



(B) 



Examples 



-Projects using multimedia experiences and 
data provided by CD-ROM for authentic 
and challenging learning 

-Local file sharing allowing students access 
to all files for communal editing and 
development 

-E-mail for inquiry collaborations 

-State network support for schools using 
the Internet for projects 



Passive Learning 



Engaged Learning 



LEARNING 
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2. Type II trajectory is B — > A. This is movement 
from engaged learning and low technology performance 
to engaged learning and high technology performance. 

3. Type m trajectory is C — > A. This is movement 
from passive learning and high technology performance 
to engaged learning and high technology performance. 

4. Type IV trajectory is D — > A. This is movement 
from passive learning and low technology performance 
to engaged learning and high technology performance* 

It is obviously counterproductive to move from D (passive learning with the least 
functional technologies) to C (passive learning with more functional, and more costly, 
technologies. If a school or group is not using technology to enhance learning and reform, 
there is little reason to suffer the higher cost for greater fimctionality. 

Movement from one Type to another depends not only on what curricular goals are 
important for the future but also on what configurations of learning and technology are in 
place now. Once new curricular goals are in place, the trajectories can be used as a very 
rough guide to what technologies are needed to move toward reaching these goals, using the 
learning and technology capacity indicators defined in the paper. 

This framework also raises the question: How can we justify the added cost and effort to 
install high performance technologies if engaged learning can be attained without technology 
or with less expensive, low-performance technologies? We would argue that high- 
performance technology adds very substantial, qualitative differences to the learning 
environment that cannot be attained without that technology. Indeed, the high-performance 
technologies such as those listed in Table 1 essentially redefine many of the parameters that 
define schooling: where learning takes place, what constitutes the learning community, who 
is the teacher, who is a learner, what the primary instructional materials and resources are, 
what the tasks and assessments are, and who produces and controls information, as well as 
what the learning context is. Such technologies also require the development of concepts and 
strategies for a new way of learning that is often nonlinear and often involves 
communications that are asynchronous and simulcast to multiple locations and very complex 
tasks and tools not previously available to students. Finally, such technologies address issues 
of equity in that they significandy redefine opportunities to learn for students who are poor 
and lack local resources and for students who are academically at risk and might otherwise 
be assigned to low-level tasks. 

When using this framework, the critical questions for schools in evaluating technology 
and the costs of technology are: What are the learning goals to which technology is applied? 
(At NCREL, we refer to these goals collectively as the vision for learning.) How are these 
learning goals moving the school toward reform? How will a technology-enhanced 
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curriculum support instruction that addresses those learning goals? How does the 
technology-enhanced approach help restructure the school to meet its plan for educational 
reform? Do the students achieve the learning goals using the technology-enhanced 
curriculum? Can the school implement cost-efficient technologies given its goals and current 
realities? Can the school extend or adapt less functional technologies so that they are more 
functional to support a global community of learners in sustained learning that is challenging 
and authentic? Are there funding strategies/partnersiiips that can reduce the cost? How can 
a school continuously plan to use technology to reach for more powerful learning goals and 
reform? 

The Importance of the Framework 

When policymakers, educators, researchers, and others view the intersection of technology 
and learning, rather than just technology by itself, it is possible to link technology to issues 
ti educational reform in important ways. 

Issues of learning, reform, and technology are perhaps more critical than they have ever 
been for several reasons. First, there was major legislation enacted in 1994 with the ficalS 
2000: Educate America Act (PL103-227) that will provide significant attention to technology. 
In fact, additional technology funding to implement Goals 2000 is expected in federal 
legislation (see Ramirez & Bell, 1994). Recent research and practice on learning indicates 
that technology can piay a critical role in changing classroom environments and school 
restructuring to promote more engaged and powerful learning. Indeed, some evidence 
suggests that technology may even accelerate these restructuring processes (see below). The 
framework proposed here could be used to design and evaluate proposals to address these 
initiatives. 

Second, high-function technologies that are needed for Categories A and B are very 
expensive. It is critical for schools and others to be informed about the differences among 
technologies and technology-based programs and how they will be used to support learning 
that is linked to the standards. It is also imperative to consider how traditional distance 
education providers can move toward high performance technology configurations and 
engaged learning. 

Third, decisions about technology, learning, and reform are critical because there is now 
an opportunity to reconfigure on a large scale how information and resources are delivered to 
schools and other agencies. Specifically, there is the opportunity to develop not only a 
national infrastructure, but also regional consortia that can distribute technology information 
and educational resources to intrastate and multistate regions, and that can provide 
regionally-based services to schools on-line and on site. Our framework proposes some 
innovative kinds of regional agencies to address these issues. 

Fourth, if it is true that some technology-enhanced programs are much more powerful 
than others in their ability to generate engaged learning (e.g., Category A in our framework), 
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then it is imperative to assure that all students have access to these powerful designs, or, at 
the very least, that access to those powerful designs is equitable. These equity issues occur 
in two different contexts. There is a significant disparity among those trained in technology 
and those less well educated. According to the Economic Report of the President (1994): 

Since the use of more-educated labor has increased in all industries, a logical explanation 
of this trend is technical change. For example, one study shows that people who work 
with personal computers earn a substantial wage premium over those who do not, and 
that this gap can account for half of the increasing gap between wages of college and 
high school graduates (p. 119). 

In terms of equity in the context of education, the concern for equity is evident in Goals 
2000, the technology legislation in progress indicated above, the focus on universal access in 
the Vice President's Agenda for Action speech (Gore, 1994), and language given below from 
the National Science Foundation (NSF). The administration and Congress are very aware of 
the disparities between affluent schools and schools for minority students and the poor and 
have generated provisions to facilitate assistance to the latter. There are provisions in the 
Goals 2000 legislation to facilitate linking the wealth of free materials in museums, libraries, 
the Internet, and other agencies to schools. There is even a strong awareness in the recent 
emphasis on universal participation from NSF (Sabelli & Barrett, 1994). There is increasing 
consensus everywhere that it is not enough to provide the technology and the connections so 
that everyone could participate; it is vital to provide ongoing professional development and 
new designs for technology-enhanced curricula so that everyone will participate. 

If we believe that all students can learn, we must overcome barriers to participation and 
use by poor and minority schools. For schools with high populations at risk, it is important 
for policymakers to (1) provide opportunities for schools and students to become informed 
about and experience the best technologies and technology-enhanced programs building on 
the ncv/ consensus emerging around learning and technology; (2) establish curricula and 
assessment that reflect engaged learning to the highest degree for students at risk; (3) give 
schools permission and time to explore and experiment with new learning and instructional 
paradigms; and (4) provide ongoing professional development to develop new learner 
outcomes, curricula, and assessment that utilize the best technologies and programs (see also 
Collins, 1993). 

Fifth, a major concern for research and policy is to anticipate that the poorest schools 
may need the most help in professional development and school restructuring. The 
framework could be used to (i) develop challenging objectives, (2) develop prototypes that 
transform free information resources into usable curricular and instructional formats to reach 
those objectives, and (3) evaluate the results of specific programs and initiatives. 

Sixth, how can we ensure that poor schools, especially those with students who are 
academically at risk, have equitable opportunities to access and use technologies related to 
successful performance in the workplace and community? There must be ways, as Kati 
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v/rL^n? y fT"?* 1 '. U S P 0 ^ 11 ^ 15 <** hel P those with the greatest challenges 
addre enH n Sf ^ rCgl0nal i"**""™ proposed in tois framework could 

f ^ t™\ways. ^ can help distribute instructional and assessment services 

Adt^otr W0Uld f m ^ W6r ^ 10 UtUi2e neW M™**™ resources 
l ow So r A ^ on f y' re 8 10nal .infrastructures could be used to generate scenarios for 
how best to rethink school economies using technology and free information resources to 

«hw a r ^ "* t0 ^» - the natiSSnStion 

v J" !!? Se ° ti0n ' WC summarize ^ research basis for this framework from Jones 
Valdez and Rasmussen (1994) and elaborate its uses and implications. Specifically we 
cover the growing consensus against traditional models of learning and definitions of 
technology effectiveness; the emerging consensus on both learning and technology learnine 
and technology interactions and school-based policy; and local, site, and naS'po^ 

2. The New Consensus 

frn^T JTL^f 6 5** b6en mCreasing ^nsensus on learning and technology emerging 
from several strands of research. First, there are strong reactions against traditional models 
of learning, traditional definitions of technology effectiveness, and traditioXXsTc^st 
effectiveness for the use of technology. Second, a very strong consensus is formhiTfroiT 
research on the importance of engaged, meaningful learning L collaboration XvT 
influent ™ d > ** recent research on learning is iSSSZ^i^ 

influenced by, our understanding of technology as a tool for learning and commutation 
» new cntena-for technology's performance. Fourth, this consensus on^g and 
technology permits us to look at the intersection of learning and technology as tSH£Sa 

fX^TJ^T t0 C T ged leaming ™ d fr0m l0w to high Performance technology 
Fifth, there are policy implications of this analysis. 

Consensus against Traditional Models and Definitions 

tZnl?* ??fr ^ v 8 8r ° Wing C ° nCem traditional models of learning based on the 
assembly model are not aligned to the needs of the 21st century. SpecificaUy toe workriacL 
and community need citizens who can think critically and strategicaSTto Zle p obTems 

di tnbuted sources and multiple perspectives, understand systems in diverse contexts and 
collaborate tocally and around the globe both synchronously and asynchronously using 

££ ^£KT ^ Nati0nal Center ° n ^on Z the Economy 8 1990; 
Keicn, 1991, (The Secretary's Commission on Necessary Skills [SCANS1 1992> These 
attributes contrast sharply with the discrete, low level basic skills and content taught iTLny 
schools. They also contrast with the transfer model of instruction, which assume! tha? toe 
teacher is the information giver and the student a passive recipient and with standardized 
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tests that assess skills sometimes only useful in schools and that often do not measure what is 
taught. 

There are also arguments from many sources against traditional definitions of technology 
effectiveness: that is, the practice of comparing the technology program to the traditional 
program in terms of student outcomes on standardized tests (see Jones, Valdez, & 
Raf mussen 1994). However, we found that the most forceful comments came m e-mail 
communications from researchers in the field in response to NCREL's request for 
information on technology effectiveness. For example, Beverly Hunter argued that 
effectiveness is not a function of the technology, but rather a function of (1) the learning 
environment and (2) the capability to do things that one could not do otherwise. Moreover, 
-Technology in support of outmoded educational systems is counterproductive (Hunter, 
personal communication, February, 1994; see also Hunter, in press). In another e-mail, 
Robert Blomeyer called the concern for standardized tests "ludicrous. He argues, 
technology works in a school not because test scores increase, but because technology 
empowers new solutions to learning and teaching needs (see also Blomeyer, 1991). 
According to Blomeyer, effectiveness must be documented in rich case studies. 

The traditional way to measure cost effectiveness for technology is to compare a 
technology-enhanced program against the traditional model of learning. Some researchers 
caution against this definition/approach. Collins (1993), for example, voices a number of 
concerns about the research on the cost effectiveness of technology: 

1 It is difficult to measure the efficiency of some technologies, such as word processing 
compared to typing. Word processing is obviously more costly, but we need to evaluate 
what would be lost by keeping the old technologies. Such cost analyses constantly 
assume that we should continue teaching the same things. 

2. Cost effectiveness data would constrain developing innovative applications of 
technology. 

3 It is not possible to control all of the variables in such comparisons. According to 
Collins providing an alternative definition of cost effectiveness is self-defeating, since 
raising questions about cost effectiveness is the wrong thing to do. Our understanding of 
Collins' perspective is that schools should explore what learning can be empowered by 
technology that is otherwise not possible through research on a small scale. Then they 
can consider issues of scaling up and dissemination. 

Herman (in press) also wonders why we keep asking the same questions when they are 
not the right questions. She sympathizes with the need for policymakers and others to have 
answers to questions about technology effectiveness on student learning, student readiness for 
workforce skills, teacher productivity using technology, and cost effectiveness. But she 
argues that we cannot find definitive answers across studies or across a given technology or 
configuration. Herman acknowledges that some examples of technology, such as Pogrow s 
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(1990), higher-order thinking skills program, HOTS, and the Jasper Series developed by the 
Cognition and Technology Group at Vanderbilt (1992b), show strong and consistent, positive 
results. However, she argues that even powerful programs might show no project effects 
due to a myriad of methodological flaws, and that it would be most unfortunate to reject 
them because measures on standardized tests showed no significant differences. Instead, she 
contends, we should develop theory-specific measures developed from a particular theory or 
model. 

In essence, what we have learned from these reactions against traditional molds is that 
we must change the questions and the process. What is most important in judging 
technology effectiveness and cost effectiveness for technology is establishing a clear vision of 
learning and goals for a school, district, or other unit. Without a vision and goals for 
learning, there are no criteria for evaluating technology effectiveness or costs. A major issue 
is the extent to which the goals and the actual technology configurations used would support 
learning and other educational reforms versus traditional models of learning and schooling 
(Ramirez & Bell, 1994), A second major issue in thinking about cost effectiveness is that it 
should not be judged by comparing the costs of a technology or technology-enhanced 
curriculum to a traditional curriculum. Rather, we should consider various technologies and 
configurations to achieve the goals and develop effective strategies for funding— an important 
equity issue because poor schools do not to do this effectively. 

Emerging Consensus on Learning 

Research on Learning, Across many strands of research there is increasing consensus on 
(1) what are the key variables of learning and instruction and (2) what defines engaged 
learning in the classroom and school. Regarding the key variables of instruction, Jones 
(1992) identified eight variables, which have been updated as follows: (1) the goals and 
metaphors that drive learning and instruction, or what we at NCREL call the vision of 
learning; (2) the tasks that ultimately define the nature and level of achievement as well as 
the curriculum; (3) the assessment principles and practice; (4) the instructional model; (5) the 
characteristics of the learning context including the nature of the learning environment, and 
the nature of the relationship among teachers and students; (6) grouping arrangements; (7) 
the learner roles; and (8) the teacher roles. These are all neutral terms that could describe 
analytically any classroom or learning environment. 

Regarding what defines learning in the classroom, there is a rich body of literature from 
various fields that must be examined. Consider, for example, theories/concepts such as the 
"new" definition of reading (e.g., The Report of the Commission on Reading, 1985); 
anchored instruction (Bransford et al., 1990); metacognition (Brown, 1978); cognitive 
apprenticeship (Collins, Brown, & Newman, 1989; see also Collins, Brown, & Holum, 
1991); multiple intelligences (Gardner, 1990); learning and teaching using the Internet 
(Hunter, in press); reciprocal thinking (Palincsar & Brown, 1984); communities of 
practice (Roupp, 1993); education and learning to think (Resnick, 1987); thinking 
curriculum (Resnick & Klopfer, 1989); cognitive flexibility (Spiro & Jehng, 1990); and 
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distributed intelligence and knowledge-building communities (e.g., Pea, 1992). Each of 
these theories of learning/instruction speaks to the eight variables and provides a rich profile 
of what engaged learning is about and what conditions are necessary or optimal in the 
learning environment to yield engaged learning. An examination of this literature reveals not 
only multiple perspectives and unique constructs but also many common assumptions. These 
commonalities exist in part because of common connections to such theorists as Dewey, 
Vygotsky, and Whitehead and in part because of major paradigm shifts in thinking in the 
disciplines and other areas. 

What are these common assumptions? What are the indicators of engaged learning when 
we examine each of the variables in the Jones (1992) framework? There are numerous 
efforts to capture these commonalities in research syntheses and principles of learning that 
cut across the research on learning and instruction; e.g., the learner-centered principles 
developed by the American Psychological Association (1993); the principles for meaningful 
classroom learning developed by Brooks and Brooks (1993); cognitive designs in education 
(Jones, 1992); and the indicators of reform instruction articulated by Means and her 
colleagues (1993). From these efforts, we have selected the indicators developed by Means 
and her colleagues as a vehicle to develop our analytic framework. Our reasons for selecting 
this analysis are: their work was based on much the same literature/theories of interest here; 
the framework was grounded in observations of successful practice and covers all of the 
variables that define instruction; and because they link the use of technology in the classroom 
to learning and educational reform. 

Specifically, Means and her colleagues developed a report for the Office of Educational 
Research and Improvement (OERI) that argues that technology should be measured by the 
extent to which technologies and technology-based programs support rich learning 
opportunities and educational reforms. Toward that end, they developed seven indicators to 
measure effective learning and reform instruction. These indicators are authentic and 
multidisciplinary tasks, performance-based assessment, interactive modes of instruction, 
heterogeneous groupings, collaborative work, student exploration, and teacher as 
facilitator. In a personal communication, Means emphasized that authentic tasks are the 
driving indicator of what she calls "reform instruction" in that all of the other features follow 
logically if authentic tasks are in place. Authentic tasks address important issues and 
problems in the real world. Such tasks usually involve collaboration, heterogeneous 
grouping, performance-based assessment, and sometimes exploration. Also there is usually 
some kind of facilitator. 

In this groundbreaking work, Means and her colleagues applied these indicators to 44 
technologies and technology-based programs in order to reveal the extent to which each one 
supports learning. We will return to this analysis at appropriate points in the section on 
learning and technology interactions. For the present, it is important to discuss the learning 
indicators in their framework. 
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We agree with the focus these researchers selected in defining each indicator and also 
with the essence of their definitions. However, the definitions provided by Means and her 
colleagues allow considerable latitude in interpretation, and it would not be too difficult for 
some users of their indicators to think that they already have these things in place when in 
fact they may not. In our descriptions of these indicators below, we have added one 
variable— the vision of learning that is the foundation for reform instruction, what we call 
engaged learning. Second, we have enhanced these definitions to reflect more of the recent 
research on learning and instruction referred to above (much of which was not available to 
Means and her colleagues) and to leave less opportunity for reductionism. Note that there is 
some amount of symmetry and r edundancy as one looks at the set of eight indicators as a 
coherent whole. That is, if the instructional model is generative, then assessments should be 
generative also. Similarly, if the learning context values diversity, then other categories of 
learning such as grouping and assessment should reflect that value also. 

1. Indicators: Vision of Engaged Learning. What does engaged learning look like when 
you see it in students? What is the vision of engaged learning that should drive the 
development of tasks, instruction, assessment, learning contexts, and student and teacher role 
definitions? Combining knowledge from research and best practice, we would define 
engaged learning in terms of four indicators (see also Jones & Fennimore, 1990; Tinzmann 
et al. 1990). First, students are responsible for their own learning; they take charge and are 
self-regulated. They define learning goals and problems that are meaningful to them; they 
have a big picture or blueprint of how specific activities relate to those goals; and, using 
standards of excellence, they evaluate how well they have achieved the goal(s). Successful, 
engaged learners also have explicit measures and criteria for assessment and self-assessment 
as well as benchmark activities, products, and/or events for checking their progress toward 
achieving their goals. They have some optional routes or strategies for attaining the goals 
and some strategies for correcting errors and redirecting themselves when the plans and 
strategies are not working. Being responsible and taking charge also means knowing one's 
strengths and weaknesses and how to deal with them both productively and constructively as 
well as how to shape and manage change. 

Second, successful, engaged learners are energized by learning. They derive excitement 
and pleasure from learning so that it is typically intrinsically motivating and yields a lifelong 
passion for solving problems; understanding; and taking the next step in their thinking, 
research, or creative production. 

Third, these learners are strategic; they know how to learn because developing and 
refining learning and problem-solving strategies are ongoing for them. This capacity for 
learning how to learn includes constructing effective mental models of knowledge and 
resources even though the information may be very complex and changing. Strategic 
learners can apply and transfer knowledge to solve problems creatively as well as make 
connections at different levels. 
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Fourth, successful, engaged learning involves being collaborative: valuing and having 
the skills to work with others. Collaborative learners understand that learning is social, that 
they must be able to see themselves and ideas as others see them, must be able to articulate 
their ideas to others, and must have empathy and be fair-minded in dealing with 
contradictory or conflicting views. They have an ability to identify the strengths of others 
and the diversity of one's own strengths and intelligences. Collaborative learners typically 
value diversity and multiple perspectives. 

2. Indicators: Tasks for Engaged Learning. As indicated above, Means and her 
colleagues indicated that in successful schools, tasks are challenging * authentic ,* and 1 
multidisciplinary.* In terms of being challenging, such tasks are typically complex and 
involve sustained amounts of time compared to tasks typically offered in schools. We find 
that tasks in the most effective schools also require students to stretch their thinking and 
often their social skills in order to be successful. Challenging tasks are authentic in a 
number of ways (see Hunter, in press). They correspond to the tasks in the home and 
workplaces of today and tomorrow. They have a close relauonship to real world problems 
and projects, build on life experiences, require in-depth work, and benefit from frequent 
collaboration. Collaboration often takes place with peers and mentors within school as well 
as with diverse people in the real world outside vof school. Tasks are also authentic when 
they represent projects and problems of relevance and interest to the learner(s). Authentic 
tasks merit authentic audiences, whether they be peers engaged in author-reader response 
groups reacting to their original writing or practitioners in the field reacting to student 
projects. Challenging, authentic tasks incorporate the knowledge of the disciplines and the 
ways in which they are used by practitioners. Students learn authentic tasks in context, 
practicing basic and advanced skills together as a means to learning the big concepts. They 
also learn something at the time it is needed for a project or problem. Authentic tasks 
usually involve and benefit greatly from multidisciplinary work. That is, challenging and 
authentic tasks often require wholly integrated instruction, which blends disciplines entirely 
into thematic or problem-based pursuits, and instruction that incorporates problem-based 
learning (PBL) and curriculum by project (CBP). 

3. Indicators: Assessment of Engaged Learningo Assessments to promote engaged 
learning involve presenting students with an authentic task, project, or investigation, and then 
observing, interviewing, and/or examining their artifacts and presentations to assess what 
they actually know and can do. In such performance-based* assessment, students construct 
the knowledge and create the artifacts that represent their learning. Ideally, students are also 
involved in generating performance criteria and are instrumental in the overall design, 
evaluation, and reporting of their assessment. In this way, the assessment is truly 
generative. The overriding purpose of assessment is to improve learning. To that end, 
assessment should closely match the goals of the curriculum; represent significant knowledge 
and enduring skills, content, and themes; and provide authentic contexts for performance. 



1 Asterisks represent indicators identified by Means and her colleagues. 
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The performance criteria should be clear, well articulated, and. a part of the student learning 
experience prior to assessment* Indeed, developing standards of excellence for learning and 
thinking is an important part of learning. Students should have internalized the performance 
criteria to the extent that they know when they have met their learning goal. Any rubrics 
used should be tailored to fit the context of the performance task. 

Performance-based assessment, at :*s best, is seamlessly interwoven with curriculum and 
instruction so that it is ongoing. Therefore, assessment should represent all meaningful 
aspects of performance. It should encompass the evaluation of individual as well as group 
efforts; self-, peer, and teacher assessment; attitudes and thinking processes; 
drafts/blueprints/artifacts of developing products as well as the final products; open-ended m 
well as structured tasks; and tasks that emphasize connections, communication, and real 
world applications. Multiple measures, e,g,, surveys, inventories, journals, illustrations, oral 
presentations, demonstrations, models, portfolios, and other artifacts of learning, are often 
needed to assess big ideas and complex learning outcomes over time. 

Issues of equity in the type of performance-based assessments described above are linked 
to issues of standards. It is critical to have equitable standards; that is, ones that apply to all 
students. Both parents and students should be familiar with those standards and be able to 
evaluate the performance of an individual or group against them, 

4. Indicators: Instructional Models and Strategies for Engaged Learning. The most 
powerful instructional models or modes are interactive sind generative by design. 
Interactive* instruction actively engages the learner. It can also be generative, encouraging 
the learner to construct and produce knowledge in meaningful and deep ways. Generative 
models of instruction have as their goal providing experiences and learning environments that 
promote deep, engaged learning in students. Further, whereas traditional models conceive of 
learning as a two-person situation (the teacher and student), generative models assume that 
learning is a three-person situation (the teacher, the student, and others) so that there is co- 
construction of knowledge— students teach others interactively and interact generatively with 
the teacher and peers. There may also be knowledge building as persons from multiple 
perspectives interact to produce shared understandings. 

Generative approaches, such as reciprocal teaching, utilize a wide range of instructional 
strategies. These instructional strategies include Socratic dialogue, individual and group 
summarizing, means of exploring multiple and differing perspectives, techniques for building 
upon prior knowledge, brainstorming and categorizing, debriefing, content-specific as well as 
general problem-solving processes, team teaching, and/or techniques for constructing mental 
models and graphic representations. All of these models and strategies encourage the learner 
to solve problems actively, conduct meaningful inquiry, engage in reflection, and build a 
repertoire of effective strategies in diverse social contexts for learning, 

5. Indicators: Learning Context for Engaged Learning. Hie classroom, when conceived 
as a knowledge-building learning community, resists fragmentation and competition and 
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enables students to learn more collaboratively and academically. Such communities not only 
develop shared understandings collaboratively, assuming that intelligence is distributed 
among the members, but also create empathetic learning environments that value diversity 
and multiple perspectives. Such communities search for strategies to build on the strengths 
of all its members. These features are especially important for learning situations in which 
there are marked differences in prior knowledge. For example, while individuals may have 
very different levels of knowledge or expertise about a topic, knowledge-building strategies, 
such as brainstorming what is known about a topic, collaboratively pool the knowledge and 
experiences of the group, thereby creating more equitable learning conditions for everyone. 

Classrooms, schools, and communities that are truly collaborative encourage all students 
to ask hard questions; define problems; take charge of the conversation when it is 
appropriate; participate in setting goals, standards, benchmarks, and assessments; have work- 
related conversations with various adults in and outside of school; and engage in 
entrepreneurial activities at times. This vision contrasts sharply with classrooms in which 
interactivity is defined as students responding enthusiastically to questions posed by the 
teacher. Collaborative classrooms also contrast with cooperative learning paradigms when 
they involve highly structured tasks and student roles defined and controlled by the teacher 
(see NCREL, #3, 1990). Collaborative work may be most powerful when it is in the context 
of flexible, learning-centered investigations that involve students with practicing professionals 
and community members. Such collaborations may occur electronically or in work outside 
the school. 

6. Indicators: Grouping for Engaged Learning, Collaborative work that is learning 
centered often involves small groups or teams of two or more students within a classroom or 
across classroom boundaries. Although the role(s) and task(s) of each student may be 
different, all members of the group collaborate to accomplish the group's goal or project. 
When a task is complex or creative in nature, it is often beneficial to use heterogeneous* 
grouping. Groups that include males and females and a mix of cultures, learning styles, 
abilities, socioeconomic status, and ages bring a wealth of background knowledge and 
differing perspectives to authentic, challenging tasks. Many teachers make use of flexible 
grouping, configuring and reconfiguring their small groups according to the purposes of 
instruction. This enables them to make frequent use of heterogeneous groups and to form 
groups, usually for short periods of time, based on common interests or needs. Flexible 
grouping, incorporating recurrent use of heterogeneous groups, is one of the most equitable 
means of grouping and ensuring increased opportunities to learn to all students, 

7. Indicators: Teacher Roles for Engaged Learning, The role of the teacher in the 
classroom has shifted and greatly expanded from the primary role of information giver to that 
of facilitator, guide, and learner. As facilitator* the teacher provides rich environments, 
learning experiences, and activities for learning, incorporating opportunities for collaborative 
work, problem solving, authentic tasks, and shared knowledge and responsibility. Such a 
collaborative classroom requires the teacher to act as a guide— a. complex and varied role that 
incorporates mediation, modeling, and coaching (see NCREL, #3, 1990), When mediating 
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student learning, the teacher is frequently adjusting the level of information and support 
needed by students and helping them to link new information to prior knowledge, refine their 
problem-solving strategies, and learn how to learn. Teacher modeling involves thinking 
aloud and demonstrating, when needed. Coaching involves giving hints or cues, providing 
feedback, refocusing student efforts, assisting students in the use of a strategy, and providing 
procedural and factual knowledge in authentic contexts, when needed. The teacher as guide 
relies heavily on active listening skills and Socratic questioning techniques. 

Given the diverse opportunities and challenges present in education, the teacher is often a 
co-learner and co-investigator along with the students. That is, as teachers and students 
participate in scientific and other investigations with practicing professionals, increasingly 
they will need to explore new frontiers and become producers of knowledge in knowledge- 
building communities. Indeed, there will be times, especially with technology, when 
students are the teachers and teachers are the learners. 

8. Indicators: Student Roles for Engaged Learning. One important student role is that of 
explorer.* Students discover concepts and connections and apply skills by interacting with 
the physical world, materials, technology, and other people. Such discovery-oriented 
exploration provides students with opportunities to make decisions while figuring out the 
components/attributes of events, objects, people, or concepts. Often students are encouraged 
to jump into an open-ended activity in order to stimulate their curiosity, become familiar 
with the instructional materials, and formulate early understandings of die task. Students can 
then reflect upon ideas and revise, reorgraize, and expand upon their understandings with 
further knowledge, exploration, and debriefing. Reflective thinking is also essential for the 
student as cognitive apprentice (e.g., Collins, Brown, & Holum, 1991). While the student is 
engaged in challenging authentic tasks and co-investigations, apprenticeship learning takes 
place when the student observes, applies, and refines through practice the thinking processes 
used by practitioners. Students can then reflect on their practice in diverse situations and 
across a range of tasks and articulate the common elements of their experiences. This will 
enable them to generalize their skills and transfer their learning to new situations. In 
cognitive apprentice roles, learning experiences are essentially formative with feedback on a 
day-to-day basis over many aspects of a complex problem or skill. There are significant 
occasions, which we predict will occur more with effective technology use, when students 
need summative (role) experiences. When students are teachers^ they integrate and represent 
holistically what they have learned intensely over a period of time, thereby yielding deep 
generative learning. Similarly, when students are producers of knowledge y they generate 
products for themselves and the community at large that synthesize and integrate knowledge 
and skills holistically. Moreover, as we shall see throughout this paper, increasingly 
stuauits, through the use of technology, are able to make what practicing professionals 
consider to be significant contributions to the workTs knowledge. 

Table 2 on the following page summarizes the indicators for each of the eight variables 
of learning and instruction. The left column shows the neutral, analytic variables for 
learning and instruction. The right column provides specific indicators of engaged learning 
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ENGAGED LEARNING AND REFORM INSTRUCTION 



Variables of Learning 
and Instruction 


indicators lor i^ngagea ^earning h 
and Reform Instruction | 


Vision of Learning 


Responsible for learning H 
Strategic | 
xinergizeo oy learning n 
Collaborative | 


Tasks 


Authentic* 
Challenging* 

Integrative/Interdisciplinary* 


Assessment 


Jrenormance-Dasea 
Generative 

Seamless and ongoing 
Equitable 


Instructional Mews 


Interactive* 
Generative 


Learning Context 


uonaDorauve 
Knowledge building 
Empathetic 


Grouping 


Heterogeneous* 

Equitable 

Flexible 


Teacher Roles 


Facilitator* I 
Guide 

Co~Learner/Co-Investigator 


Student Roles 


Explorer* 

Cognitive apprentice 

Teacher 

Producer 
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* Shows indicators provides by Means and her colleagues (1993). 
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for each variable, combining the indicators given by Means and her colleagues and those 
given by us. In the next section, we develop analytic descriptors for technology and 
indicators for high performance technologies that promote engaged learning and instruction. 

Emerging Consensus on Technology Performance 

Looking at technology through the lens of learning, we developed six neutral variables of 
technology performance that are descriptive and analytic. These variables are (1) school 
access to diverse technologies and resources both beyond the school and within a given 
classroom; (2) operability; (3) location and direction of resources— whether the technology 
has a central source or has distributed resources; (4) capacity of the technology or program 
to engage students in challenging learning; (5) ease of use; (6) functionality, which is the 
capacity of the technology to prepare students for a diversity of technology functions. 

What does high performance technology look like when we consider each of these 
variables? How can technology be designed to promote engaged learning? Again, as was 
the case with learning, this question could be answered in somewhat different ways by 
different technology specialists and researchers. In feet, there is now a burgeoning literature 
on design features of technology that promote meaningful learning and collaboration (e.g., 
Cognition ana Technology Group at Vanderbilt, 1991a, 1992a; in press; Cunningham, 1991; 
Duffy & Jonassen, 1991; Hawkins & Collins, 1992; Knuth & Brush, 1990; Newman, 1992; 
Pea, 1992; Schank, 1990, 1991; Spiro, Feltovitch, Jacobson, & Coulson, 1991; Wilson & 
Tally, 1991). 

Despite the diversity and richness of specific concepts in this literature, there is strong 
consensus in this research community that technology artifacts such as tools and technology- 
enhanced programs can be designed to promote engaged learning. Moreover, there are many 
features of technology and programs that continually recur in diverse contexts and/or have 
been developed over many years by a specific research group. Using this literature as a 
backdrop, we defined indicators within each of the six technology performances that would 
yield high performance (vs. low performance) and would promote engaged learning and 
reform instruction. Please note that in this component of our framework, there is necessarily 
a certain amount of o n srlap, redundancy, and interrelationships among indicators. For 
example, connectivity to the Internet implies distributed resources. 

1. Indicators: Access. A technology or technology-enhanced program has high 
performance in terms of access when it has connectivity, ubiquity, and interconnectivity. 
Connectivity refers to the capability of technology to access rich resources within and beyond 
the school because it is connected to those resources. "Last-mile connections" from the 
school to a telecommunications source must be in place if schools are to access the wealth of 
free and low-cost resources on the information highway. Ubiquity means that computers, 
printers, media technologies, and other equipment must be everywhere within the district and 
school so that all teachers and students can access and use them as tools to solve problems, 
communicate, collaborate, and exchange data. This does not mean that every student must 
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have a computer; that would be ideal* It does mean that having a computer or multimedia 
laboratory in every school is not ubiquitous because students and teachers have to go 
someplace and perhaps experience long delays to use the equipment. Under these conditions, 
users cannot use the equipment as an everyday tool. What is needed is to network a critical 
mass of computers and other equipment, especially printers, throughout the school so that 
teachers and students can use them when they need them. Interactivity refers to the 
interaction that occurs when students and teachers actually communicate and collaborate in 
diverse ways (exchanging data in different formats, and publishing). 

In addition to these three features, there is the issue of what users access and sihQ has 
access to the best and most extensive resources. A system can be connected and interactive 
but not all students are accessing the quality resources. For example, a system may have 
home-school connections but not to the local library system or the Internet. Or schools may 
have connections to the library system and the Internet, but only the students in the gifted 
classes or the magnet schools are really set up with instructions and training about how to 
use those connections effectively, while low-achieving students are working with programs 
focused on low-level objectives. Also, poor schools may not be well informed about the 
more powerful connections and programs, so they always get a busy signal or their software 
may be so poorly connected and configured that it is too cumbersome to use frequently* 
There has to be a powerful design for equitable use of the technology, program, or 
configuration to address these issues. That is, technology use in schools should be designed 
and implemented so that all students have access to rich and challenging learning 
opportunities and instruction that is interactive and generative. It is this design feature that 
promotes opportunities for engaged learning for all. 

2. Indicators: Operability. A technology or program has maximum performance in terms 
of operability when it has interoperability. According to recent legislation (e.g., Congress of 
the United States, 1994), interoperability is the capacity "to easily exchange data with, and 
connect to, other hardware and software in order to provide the greatest access for all 
students." To achieve this, it is necessary to have open architecture. This feature allows 
users to access data using different (third party) hardware and software; it also allc *'s users 
to modify the system, sometimes dramatically (e.g., t^e capability to add one's own template 
to a spreadsheet program). Open architecture means that software at major 
hardware/software outlets has "shrinkwrap availability"— a common metaphor in the field. 
Interoperability also requires transparency (moving from one format or program to another 
easily and unobtrusively). This means that users are essentially unaware of the procedures 
used by the hardware and software for changing programs and multitasking (allowing users 
to be working on several tasks at once within the same system). 

Technologies or programs that have open architecture and transparency are likely to 
promote engaged learning because they allow the learner to spend maximal time and energy 
enjoying and using the resources they access, rather than spending the time and energy in the 
process of learning to use the technology and/or doing complex and time-consuming 
procedures to move from one program or format to another. Moreover, these capacities 
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promote learning because they allow the user access to more resources, an issue we will 
discuss below in reference to other indicators* 

3, Indicators: Organization of Resources. Indicators in this category pertain to questions 
such as, Where is the information/data stored? How are resources connected? How do new 
resources get into the system? Is the transmission asymmetrical (from one source to another) 
or symmetrical (two-way transmission capability)? Is information flow one way from a 
central source to others (asymmetrical) or two ways (symmetrical)? Who is in charge? 

In some schools and technology programs (1) information is highly centralized, typically 
in mainframes or other centralized servers, (2) students may use "dumb terminals," which 
have few capabilities, and/or (3) users can share files from other users within the 
system— within the same building or district. In such systems, most of the information is 
asymmetrical; it flows in one direction— from the system to the users. The system operator 
is in charge of what information and resources are entered into the system, when it is entered 
and distributed to others, and so on. An example is a distance education course that is 
provided by a university teacher whose role includes such things as providing most of the 
information, coordinating who asks and answers questions, and defining the student 
assignments and products. Another example is what we call an "electronic basal," which 
provides highly structured information, instruction, and assessments for students; makes 
decisions about which unit or module to work through next; and controls when and how the 
student responds to the information presented by the system. 

Centralized systems are likely to inhibit learning to the extent that they employ the 
transfer model of learning and instruction. This model assumes that the central source 
" contains" most of the important information to be learned and that it is the job of the 
student to transfer the information from the central source to the user's location and "learn" 
it. Such systems may be powerful technologies that offer rich resources such as a 
multimedia encyclopedia, options to use an array of media technologies including video, and 
efficient management systems for assessment and record keeping. From this perspective, 
central source systems would be high performance. However, the high performance and 
power may be very limited, relative to distributed systems. Further, learning may not be 
very engaged because the model of learning is essentially a one-way transmission model and 
the objectives are likely to be low level, focusing on basic skills. 

In contrast to these centralized and relatively closed resource systems, distributed 
resources are organized very differently (see Newman, 1992). Specifically, they assume (1) 
that intelligence does not reside in individuals but is socially constructed through 
collaborative efforts and (2) that the resources that shape and enable activities (to build 
socially constructed knowledge) are distributed across people, environments, and situations 
(Pea, 1993, p. 50), This feature allows users to access resources (1) from anywhere in a 
local system (Local Area Networks, LANs) and (2) from anywhere external to the system 
such as from the Internet (Wide Area Networks, WANs, or world wide web, WWW). Thus, 
systems that provide only a LAN are considerably more limited than systems providing 
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WANs and WWWs. It is clear that these networked, open systems are designed for two-way 
transmissions and user contributions such that the information is reusable. That is, 
information, products, and services can be contributed to the system from multiple sources 
such that large numbers of users can share common data sets or problem spaces. Such 
systems have distributed logic in that much of the logic of preparing documents and artifacts 
for the systems resides in the user who must comprehend and build the linkages within and 
among documents or data sets. Thus, users must understand the logic of how the resources 
are distributed. They are in control of when they make contributions and what those 
contributions are. 

Moreover, such systems typically involve components or tools that are designed for 
collaborative projects and co-investigations. For example, on-line conferences and bulletin 
boards with asynchronous communications capability, access to remote files and joint 
products, and the capability to communicate synchronously with two or more computers 
accessing the same file at the same time— all these promote collaboration. Other examples 
include programs that help groups form consensus, brainstorm, outline, develop plans, 
schedule meetings, monitor programs on group objectives, and develop joint products. Such 
systems inherently afford the user the opportunity to examine data, problems, and decisions 
from multiple perspectives. All of these capabilities facilitate developing knowledge-building 
communities in which many users converse to develop common understandings, products, 
and services (see Pea, 1992; Scardamalia & Bereiter, 1992). 

Major technical issues that have arisen with regard to distributed resources are, How 
powerful is the system? Can it communicate in diverse media (audio, video, print, and 
virtual reality)? How many users can use the same resource without delays and loss of 
quality to the data? How many tools can the system support for simultaneous use? 

4. Indicators: Design Features for Engaged Learning. High performance technologies 
and technology-enhanced progiams can be. designed or set up locally to promote engaged 
learning. One such design feature is for the software itself to provide challenging tasks, 
opportunities, and experiences or access to those things. This refers to the system's 
capacity (a) to provide complex problems and cases, links to challenging curricula, and 
unique repositories from museums and libraries; opportunities to examine contrasting events 
or data sets; (b) to access experts, peers, community members, and/or other learners who 
can guide, mentor, tutor, mediate, broker, share, inform, and involve users in productive and 
meaningful ways; (c) to use the richest media resources— images, audio, video, 3-D, virtual 
reality— for data manipulation and for presentations; and (d) to provide tools for interactive 
browsing, searching, and authoring. 

A second design feature is for the software to allow students to learn by doing, to situate 
the learning in captivating and challenging activities. Thus, tools such as authentic goals- 
based scenarios, problem-based learning, problems anchored or embedded in challenging 
narrative situations, and simulations would provide the user with experiences to develop 
expertise using real-world problems and resources. Such tools allow the user to plan, 
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reflect, make decisions, experience the consequences of actions, change directions, and 
examine alternative solutions and assumptions* 

A third indicator of designs for learning is the extent to which the system provides 
guided participation (Collins, Brown, & Holum, 1991; Merrill, et al, 1993; Pea, 1993), 
This is the capacity for such software features such as Socratic questioning, intelligent 
tutoring, diagnosis and guided analysis of errors, and adaptation of the system to respond to 
student responses, to customize the content to particular interests or learning styles* Another 
way to guide participation is to make explicit what is typically implicit (e.g., Edelson, et al., 
1994; Pea, Edelson, & Gomez, 1994). To explain: Thinking is an activity that is often 
covert and/or full of implicit references and assumptions. There are various ways that tools 
can help students see how practicing professionals and others think. One way is to build 
intelligent tools, such as wizards (that help users work through a set of complex procedures), 
embedded questions or prompts, and coaches. These tools provide the learner with 
opportunities to anticipate problems, subsequent events, and others' thoughts such as in 
cartoon bubbles. Another way is to provide powerful ways for students to have file sharing 
so that students can make their own thinking more explicit in their writing and they can see 
how others read and respond to their work. 

Another design indicator to promote engaged learning is the capacity of the system to 
provide information that is just in time and just enough. Hypertext, for example, provides 
for nonlinear learning and thinking and multiple points of entry so that the user can quickly 
access specified chunks e? information. Hypertext, and other tools, may also be designed for 
users with different levels of expertise (introductory and advanced) and for infonn&tion 
access, help commands, and user control. Finally, high performance on this indicator means 
designing a system so that persons who have little time and/or immediate, pressing problems 
have easy access to simplified and useful information, while persons with time for reflection 
and exploration have access to more complex and rich information. 

5. Indicators: Ease of Use* High performance with regard to ease of use refers to several 
features. User friendliness and effective help opportunities that are truly informative, well 
organized, and context specific are essential. Speed of processing and operations with 
feedback provided for all delays make a system easier to use than a system that is slow and 
does not provide such feedback. User control means that the user can access tools, 
information resources, experiences, and opportunities on demand and use them to solve 
problems, make decisions, and create products. This design feature promotes intrinsic 
motivaUun and exploration as well as being responsible for learning. Training and support 
to use the technology, as well as to apply it to solve problems, create products, and so on, 
are vital for all users. Further, training and support should be available both locally and 
from remote locations; and it should provide quality training and support resources. 

These ease-of-use features help the learner access the resources and learning experiences 
offered by the system. Whether the learning that results is engaged or passive depends on 
the design and other features of the technology. 
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6. Indicator: Functionality. One indicator of high functionality is opportunities for 
students to use media technologies such as color printers, video cameras and video editing 
equipment, facsimile machines, audio recording and editing, and various graphics* A second 
indicator of high functionality refers to the capacity of the technology or technology- 
enhanced program to prepare learners to use the diversity of tools that are basic to learning 
and working in the 21st century. This would include teaching students how to use such basic 
or generic tools as databases, spreadsheets, and word processing that are needed for learning 
and for use in the workplace and community. High functionality would also mean providing 
students with options to learn how to use context-specific technologies such as radiology in 
medicine and biology, or tools such as sonar for oceanographic research. 

Third, it is important that students develop programming and authoring skills , not as an 
end it itself but as a means to doing meaningful work in school, at home, and in the 
community. Moreover, such skills should be taught more in the vein of learning how to 
learn because specific programming languages and authoring nrograms are being developed 
continually, and students need to develop criteria for selecting them as well as some facility 
to learn them. 

A fourth indicator is the capacity of the software to develop skills related to project 
design and implementation such as setting goals and benchmarks, creating and monitoring 
budgets, conducting research and development, preparing analyses and presentations, 
developing dissemination skills, and marketing. All of these activities, which are 
increasingly widespread in project-centered schools, the workplace, and the community, may 
be accomplished using technology. Developing such skills also means having a good working 
knowledge of what the Internet is all about and how to navigate it for resources that are 
needed foi projects. Thus, students need to learn about the Internet not as a replacement for 
programming language taught as an end in itself but as a resource for a project or tool. 

A fifth indicator of functionality is the extent to which a technology or technology 
program prepares students to use tools that create new tools. This refers to opportunities to 
use tools such as wizards and Mosaic as well as opportunities to learn programming and 
authoring skills to create new programs and tools for others to use. This contrasts sharply 
with traditional approaches to technology which might, for example, teach students outmoded 
programming languages as an end in itself. 

Table 3 on the following page summarizes the variables of technology performance and 
the indicators for high technology performance. It is important to note from the outset that 
not all of these categories apply to all technologies. For example, it i" possible to have high 
performance tools that promote engaged learning and educational reform but which may or 
may not have interoperability, connectivity to other resources, and distributed resources. 
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TABLE 3 



HIGH PERFORMANCE TECHNOLOGY AND ENGAGED LEARNING 



Variables of Technology 
Performance 


Indicators for High Technology Performance -> 
Engaged Learning 


Access 


i^onnecuvuy 

Ubiquity 

Interactivity 

Design for equitable use 


Operability 


mteroperaDuiiy 
Open architecture 
Transparency 


Resource Organization 


Distributed resources and logic/intelligence 
User contributions 

Uwolgn IOr C011<UWIaUVC piUJCwlo 


Engagement 


Opportunities for challenging tasks and experiences 
Opportunities to learn by doing | 
Guided participation and intelligent tutoring | 
Information just in time and just enough | 


Ease of Use 


Effective helps | 
User friendly | 
User control | 
Training and support | 
Speed 


Functionality 


Use of multimedia technologies 

Use of generic tools | 

Use of context-specific (work-specific) tools 

Programming/authoring skills 

Design and project implementation skills 

Use of tools that are used to make tools/programs 



Copyright ° 1994 North Central Regional Educational Laboratory 
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Learning and Technology Interactions and School-Based Policy 

Now that we have rich indicators for engaged learning and high performance technologies, 
we can return to the argument made by Means and her colleagues (1993). Specifically, 
technologies and technology-enhanced programs should be measured by the extent to wMch 
they support engaged learning and educational reform. There are two ways that one could 
analyze the intersection between learning and technology. First, one could develop tools to 
score the indicators systematically in various ways. Second, one could apply the concepts of 
engaged learning and high performance technologies to examine the intersection of learning 
and technology in broad terms. Each of these approaches is described below. 

Tools for Schools and Others. We have developed a series of tools that link learning 
and technology to local school policies. These tools may be used by schools and others to 
serve various functions. They could be used to plan and design a program, describe and 
improve an existing one, or compare two or more programs under consideration for 
purchase. They could also be used by researchers, policymakers, evaluators, teacher 
educators and their students, agencies developing standards for similar purposes, and 
commercial vendors. 

The tools we developed require the user to reflect on each indicator for learning and 
technology, to provide a score for each one, and to plot the scores in various ways. 
Specific,? liy, the scoring system we have developed allows users to indicate the level of 
involvement or development with regard to that indicator at the practice level and at the 
policy level. To elaborate what is meant by practice and policy, consider the following 
example comparing two hypothetical schools and their use of authentic tasks. 

Example: School A has just begun developing such tasks through a very powerful 
technology program they are piloting (learning practice). They have implemented the 
program as part of their new mission to become aligned with the learning and technology 
needs of the 21st century (learning and technology policy). In contrast, many teachers at 
school B have been developing authentic tasks through the school's assessment program 
(learning policy), and this development is a major strength. School B has almost no 
technology in place (technology practice), but it has become a major agenda in their 
school improvement plan (technology policy). 

Tables 4, 5, and 6 in Appendix B are tools that others can use to design, describe, and 
evaluate various technologies and technology-enhanced programs or curricula. Table 4 
allows users to identify their Current Realities in terms of engaged learning and high 
performance technologies in two dimensions. First, users identify which of the indicators 
they have in place in classrooms now and to what degree. Then, they icentify which of the 
indicators they value to the extent that they are incorporated in some way into a specific 
shared document or work, such as a mission statement, curriculum framework, strategic 
plan, design for a preferred future, assessment system, professional program, and the like. 
The combined scores then reflect current realities at the classroom level and at the policy 
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level for engaged learning and high performance technologies. Users plot the intersection of 
Current Realities for Engaged Learning and Current Realities for High Technology 
Performance on Table 6, marking it with dotted lines. 

Table 5 allows users to reflect on Future Goals for learning and technology using the 
information from Tables 4 and 6. Then they decide where they want to invest additional 
resources. For example, if use - A has developed very high performance technologies but has 
limited capacities for engaged learning, then he or she may want to invest more heavily in 
developing capacities for engaged learning. Suppose, however, that a school has strong 
values/policies for engaged learning, but not a lot of indicators in place just now, and not a 
lot going for them in technology— either in the classroom or in policy. This school might 
want to look at the possibility of developing school policies on technology that can interact 
with the policies on learning and develop the capacity for engaged learning using technology. 
A different scenario might be that a school is highly developed in terms of learning capacities 
in the classroom and policies but has not become very involved in technology. Such a school 
might use the tools here to strengthen their policies on learning and let these policies and 
classroom capaciti's drive decisions and developments for technology. Reflecting in this 
way, users can use Table 5 as a design tool to generate new school-level policies and 
capacities for the Future Goals. Then these Future Goals can be plotted on Table 6 using a 
solid line. Thus, Tables 5 and 6 serve as tools for planning, design, and evaluation. 

Tables 7 and 8 in Appendix C were developed to assist schools and policymakers in 
comparing two technologies or two technology-enhanced programs. These tables allow the 
user to mark all of the indicators for learning and technology respectively for both programs. 
Table 8 may be used to plot the two programs and see the differences graphically. 

Technologies: Their Capacities, Designs, and Uses in Schools. In the next section, 
we examine the intersection of learning, technology, and policy in general terms. This 
analysis assumes that learning is an interaction of the technology capacities as well as its 
design and the uses of the technology in the learning environment. A technology can be high 
performance or low performance, but not be used effectively to maximize engaged learning. 
Our intent is threefold: (a) to categorize how the technology is typically used in schools; (b) 
to consider how its design and/or uses in school could be configured to move more toward 
very engaged learning and high performance (category A in Table 1); and (c) to examine 
state and national policy implications of this analysis. 

In this examination, we will use the broad categories and trajectories presented in Table 
1 that describe engaged learning and high performance technologies. We will also refer to 
the 26 learning indicators in Table 2 and the 25 indicators in Table 3 where appropriate, but 
our task at this time is not to analyze e-mail, computer-based approaches, and distance 
education and systematically work through each indicator for each approach. The intent here 
is first to generate a general idea of how each type of technology could be described when it 
is examined from the perspective of engaged learning and high performance technology. 
Then we will highlight some exemplary approaches/programs in each technology area. 
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L E-inaiL E-mail by itself is an inherently low-performance technology because it has only 
one function— to communicate. Therefore, issues of access, operability, resource 
distribution, and many of the design for learning features really do not apply- However, 
e-mail can be used effectively in schools to provide access to rich learning experiences, 
such as communicating with a tutor or mentor, and to promote collaborative work. E- 
mail inherently involves some degree of interactivity and exploration, but some 
interactions and explorations are more powerful than others. When students use e-mail 
to write informally to pen pals in another state, this is a good beginning- Some 
interesting and perhaps even powerful learning experiences may result, but they are 
episodic and unplanned. Teachers could use the same e-mail system to explore deeply 
complex cultural and linguistic issues or to solve problems with distant peers over a 
sustained period of time (e,g M Kidsnet, 1991); to communicate with practicing 
professionals and community members; and/or to conduct collaborative projects that will 
yield sustained, engaged learning and collaboration for challenging academic objectives. 

2. Computer-driven software and approaches* Similarly, computer-driven software and 
approaches must be considered as an interaction of the technology design and the 
learning context or purpose. Computer-based instruction (CBI) used for drill and 
practice on traditional objectives is passive learning and technology. For example, 
Means and her colleagues (.1993) argue that CBI focuses on teachers transmitting 
information to students, so that students are passive rather than active learners. They 
add that learning is divided into discrete content areas that require only "simple" 
responses from students, and that the focus is on drill and practice (see also Rockman, 
1991). Additionally, Means, et aL, raise equity issues: "Students at risk of academic 
failure— often seen as lacking in basic skills and therefore unable wO acquire advanced 
skills— become logical candidates for CBI drill-and-practice instruction. Recent research 
and thinking on the needs of disadvantaged students stress a different need . . . 
opportunities to acquire advanced thinking skills and . • • basic skills within the context 
of complex, meaningful problems." (See also Means, Chelemer, & Knapp, 1991, for an 
anthology on this topic) 

However, there are numerous other computer-based technologies derived from artificial 
intelligence (AI) and research from cognitive science that promote engaged learning. 
Such systems are designed to help learners think through complex, authentic problems; 
take charge of their own learning; and/or develop products for teaching or use in the real 
world. Such systems may use integrated media to: 

• Provide sophisticated expert systems for learning very complex concepts and 
procedures 

• Help students develop advanced skills such as reasoning, summarizing, high level 
self-questioning, and reflection 

• Diagnose and reduce student errors as well as remediate specific learning problems 
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• Adjust or adapt the level and sequence of problems based on student performances; 
suggest directions for new learning 

• Stimulate the use of emerging technologies and decision-making to address complex 
real-world problems and issues, thereby providing learning by doing and guided 
participation 

Consider two tools developed by the Institute for the Learning Sciences at Northwestern 
University. Sickle Cell Counselor, for example, was developed in collaboration with the 
Museum of Science and Industry in Chicago. This high performance AI tool was 
designed for the informal learning that is characteristic of museum settings. Users, 
therefore, are all ages, and only those interested in finding out more about sickle cell 
anemia would stop and use this tool. The program allows users to access vitally 
important health information by asking experts, by conducting laboratory tests, by 
interacting with "patients," and by seeing the consequences of communications to the 
patients (see Bell, Bareiss, & Beckwith, 1994; Institute for the Learning Sciences, 1994; 
Schank, 1991). Although Movie Reader cannot adapt to the responses of students and has 
fewer high performance indicators, it is nonetheless a very useful tool that embeds 
critical comprehension questions for students in video texts and allows teachers to 
generate additional questions to improve comprehension (Holum & Beckwith, 1993). 
This tool helps students to be responsible for learning, think strategically, collaborate, 
and generate deeper comprehension, thereby energizing their learning. From the 
teacher's perspective, the tool has facilitation and some guiding functions, with elements 
of cognitive apprenticeship in that the tool essentially coaches the students. % 

Other tools assist teachers and principals in various functions. For instance, the Illinois 
State Board of Education (1994) developed software for school improvement planning in 
partnership with NCREL. This software allows schools to enter student outcome 
specifications and assessments as well as other school improvement reports that are 
correlated to statewide school improvement plans. This software also assists educators 
through the very complex task of developing learner outcomes and assessments that are 
linked to national standards and meet state standards. The tool also assists users to think 
through the process of gathering and analyzing data about the assessments; to determine 
the quality of the assessments and assessment data; and to report the results to the 
community and the state. This tool will aid both the policymakers in seeing the "big 
picture" for each school and the classroom teachers in using the data to improve 
curriculum and instruction. 

Also at NCREL, Knuth and others are developing software for School Development 
Resource Systems. Each system has (1) a library component whereby users can access 
abstracts for a range of topics related to school restructuring to promote learning; (2) a 
video-based component for classroom teachers to view and analyze various dimensions of 
expert teachers teaching a whole lesson, using a powerful database of print materials that 
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include lesson plans and articles as well as resource contacts; (3) a multimedia database 
using Mosaic that is designed to help users make decisions on critical issues. 

Additionally, The Cognition and Technology Group at Vanderbilt University is 
developing tools for teachers in inservice and preservice contexts. One of the hallmarks 
of this research is the use of contrasting lessons on video discs, mostly in science and 
mathematics, to stimulate reflection among preservice students. However, Goldman and 
her colleagues found that it was very effective to have groups of students create their 
own integrated media presentations using pairs of contrasting mathematics lessons 
(Barron & Goldman, in press). Risko (1992) uses video-based cases to broaden 
preservice teachers' perspectives and expectations by having them explore multiple 
sources of information that influence teaching students with reading difficulties. (See 
also Means et al. (1993) for a discussion of additional tools to support other teacher 
functions.) 

Increasingly, computer technologies will "read" and "think" like humans, providing 
Socratic dialogue, analysis of human thinking, and interactions that are capable of 
tracking and responding to complex lines of inquiry (Beck, 1994; Schank, 1990; Schank 
& Edelson, 1990; Kumar, Smith, Helgeson, & White, 1994). There is also the capacity 
that computers and integrated media have to promote opportunities for learners to take 
charge of their learning (Papert, 1993). In terms of evaluating computer-based 
technologies, this means that the evaluation must consider the intelligence or thinking the 
computer can perform as well as its purpose or use in achieving a given instructional and 
learner goal. 

3. Integrated laming Systems. Integrated Learning Systems (ILS) are very high 
performance technologies, but in terms of their capacity to promote engaged learning, 
typically they are essentially electronic basals using traditional tasks and assessments, 
student and teacher roles, and approaches to instruction. They are designed to provide 
lessons and assessments targeted to basic skills required by traditional school objectives. 
ILSs are popular in part because they provide lessons, extensive teacher manuals, and 
assessments that are definitely aligned with each other within the ILS (though they may 
or may not be well-aligned to the objectives of all such schools). Some programs are 
interdisciplinary and include multimedia encyclopedias. They also provide inservice 
training for the system and the content of the program, easy-to-use and time-saving 
management systems, and good support for technology. Computers may be linked in 
local area networks (LANs). The Jostens Basic Learning Systems is an example, and 
they produce numerous profiles of successful implementations, mostly showing gains on 
standardized tests (promotional literature). 

Contrast this use of high performance technology with other high performance 
technologies configured to network schools and communities in wide area networks 
(WANs). Consider, for example, collaborative projects designed to study and manipulate 
actual images from NASA and to communicate with practicing scientists using the 
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Internet. In such systems, the technology hardware and software are configured for 
access to authentic, generative, and challenging data; for learning by doing; sometimes 
for guided participation and intelligent tutoring; as well as for user contributions and user 
control. The second example reflects the high end of numerous indicators including 
authentic, challenging tasks; instruction that is interactive and deeply generative; learning 
contexts that involve knowledge building; and the characteristics of teacher and student 
roles that promote engaged learning. It is important to distinguish between ILS and more 
open systems because many schools that have ILS believe that because they are high 
performance technologies, they therefore provide engaged learning and access to very 
rich resources. 

Newman (1992) has criticized the way that most schools organize local area networks 
(LANs), especially ILS. "Although networking technology has tremendous potential to 
support school restructuring, for the most part, it has been counter productive— or at best 
irrelevant— to any significant change" (p. 49). While most schools have the modems that 
would link them to other sources of information and educational resources, network 
technology in most schools is set up primarily to download instructional materials from a 
central repository to isolated classrooms. The problem here is not the LAN itself, but 
the way it is designed to provide learning. 

The core of the problem, according to Newman, is that ILSs are configured to provide 
information from a central source using local area networks for communication within a 
school and between schools using that ELS. That is, local area networks (LANs) are not 
connected to wide area networks (WANs) and distributed resources that could provide a 
wealth of external resources as well as opportunities for active learning and 
communication. Newman contrasts this model of learning with descriptions of Earth 
Lab, which allows students and teachers to access school resources throughout the world 
and engages students across the school in such tasks as editing the school newspaper. 
This distributed system also helps teachers to collaborate to design integrated, 
multidisciplinary curricula. Students and teachers subscribe to such services as weather 
reports, electronic mail, and bibliographic retrieval. 

At the same time, Means et al. (1993) recognize that there is an emerging trend for ILSs 
to offer schools the capability to access "third-party" software and therefore provide 
more instructional options. We are also aware of ILS companies that offer some 
opportunities for engaged learning by providing teachers with powerful multimedia 
production capabilities to create their own curricular models as well as curriculum 
development services. For example, a few ILSs are developing (1) networking outside 
the system, (2) more powerful instructional designs that focus on authentic tasks (in 
addition to their "low end" lessons addressed to basic skills), and (3) ongoing 
professional development support, including curriculum development services. One ILS 
actually owns four satellites that allow two-way video communications among schools 
and the ILS as well as video cameras for all schools. Users can create their own video 
demos, share them with others, seek advice from expert teachers at other schools, and 
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contribute to the video resources of the system, making it self-renewing. This ILS also 
assists schools to create new curricular units. 

So what is the bottom line on the value of CBI and BLSs? The critics are saying that 
closed system electronic models such as the typical CBI and ILS that support traditional 
learner goals, curriculum, instruction, and assessment are fundamentally not much 
improved over traditional, nontechnology instructional models. In particular, the new 
research on learning is saying that traditional models of learning and schooling are not 
adaptive to the needs of modern society; we need to develop new paradigms of learning 
for technology designs. What these critics of CBI and ILS are saying is that to the extent 
these technologies support traditional teaching learning, they are misaligned with 
educational reform and the needs of the 21st century. 

4. Distance education technologies* Similar analyses may be made for the interaction of 
learning and traditional distance education technologies. The majority of distance 
education courses still address traditional instructional goals. Indeed, "as of the current 
moment, the primary technology used for K-12 learning distance programs is based on an 
instructional television model which has been around as early as the 1930s. " (Westrum, 
1994, p. 2). Specifically, this refers to the use of one-way video with two-way audio, 
two-way video and audio, or two-way audio and/or audio graphics. The major distance 
education providers for students and for professional development include the Public 
Broadcasting System (PBS); TY-IN Network in Webster, Texas; the Ohio-based Satellite 
Educational Resources Consortium (SERC); the Arts and Sciences Telecommunications 
Service of Oklahoma State University (AST/OSU); and 11 of the 13 Star Schools 
Projects, as well as the satellite-based Star Schools Projects. Additionally, many states 
such as South Carolina, Virginia, West Virginia, and Wisconsin have developed rich 
resources/courses for statewide access. 

There are numerous studies that demonstrate, according to Russell (1992), that regardless 
of the quality of the production or the specific technologies used, students learn equally 
well with each technology and learn as well as their on-campus, face-to-face counterparts 
(see also Shavelson, Webb, & Hotta, 1987), There are two crucial issues here. First, 
what have these traditional distance education models accomplished? These analyses 
indicate that such models have achieved two of their major objectives: many students 
would have no instruction or very * ? Tiited instruction without them, ami this instjuction is 
generally recognized as being equivalent to the instruction in regular classrooms. This is 
particularly true for the many specialized and advanced placement courses that enable a 
large number of students to enter college, an opportunity not otherwise possible. 
Similarly, distance education technologies provide access to many collections of rare 
documents and artifacts otherwise not available to remote locations (Office of Technology 
Assessment, 1988; 1989). 

Second, there are issues related to the idea of making distance education technologies 
more capable and more powerful, which would of course make them more expensive. 
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Russell (1992), for example, sees this as a problem* He asks why we should invest in 
more expensive technologies if less expensive technologies can accomplish the same 
goal? 

We propose that this is the wrong question. What is the value of developing or 
supporting inexpensive or expensive technologies if they do not promote engaged 
learning? This is true whether the program provides a rare collection or instruction to 
rural schools that would otherwise not be possible* What is critical in the next 
generation of distance education paradigms is developing and supporting the technologies 
and models of instruction so that learning is interactive and generative; learning contexts 
are more focused on knowledge building; students are engaged in authentic, challenging 
tasks and have more control over their learning; teachers serve as facilitators, guides, 
and co-investigators; and schools may access distributed resources the world over. In 
fact, both the providers and the models are evolving toward these ends. Specifically: 

• Some traditional distance education providers are giving more priority to recent 
research on learning and educational reform. Thompson, Simonson, and Hargrave 
(1994), for example, summarizes literature and presentations from a conference of 
providers. Factors examined included cognitive and affective aspects of learning, 
collaborative learning formats, distance education as a system involving many 
subsystems, and distance education in the context of school restructuring, 

• More distance education programs are using interactive and networked designs. 
These designs utilize as part of the instruction computers, telephones, video by 
telephone, facsimiles, audiographics, and other technologies. Particularly exciting are 
opportunities for students and teachers to take expeditions electronically such as the 
JASON series, JASON is the acronym for a portfolio of satellite-based projects that 
follow the scientific activities of the world-renowned oceanographer and archeologist, 
Robert Ballard, He takes cameras into oceans, caves, rain forests, coral reefs, and 
the Mayan ruins of Belize, Students and teachers may communicate directly with 
him, his scientific teams, and other project participants via video teleconferences and 
computers. The JASON Foundation provides bulletin boards, software to download 
text files and data from project sites, and instructional materials that provide data 
synthesized by the scientists, challenging problems, biographical information about the 
Argonauts, and information about procedures for using the various technology 
components, PBS has also used many of these features in some of its distance 
education programs. Moreover, some of the satellite-based Star Schools projects and 
other distance learning providers are also using computer networks, more 
collaborative learning, and authentic tasks as well. 

5. Distance Learning Using the Internet. Increasingly, we are beginning to move from a 
focus on distance education programs and technologies to distance learning (Karim, 
1994), In part this is because promoters of engaged learning are asking what distance 
education technologies facilitate engaged learning. In part this movement is emerging 
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from the trend to understand information and data as resources, not just programs, 
spurred on greatly by the development of \hi National Information Infrastructure (Nil) 
and the concept of using the Internet as a major vehicle for distance learning at all 
levels. Indeed, Nil was designed to make "the best schools, teachers, and courses . . . 
available to all students, without regard to gender, distance, resources, or disability" 
(Gore, 1994). This dream will be enhanced and accelerated by software such as Mosaic 
that has the capacity to transport video and voice images. The dream will also be realized 
by access to digital libraries that offer collections of art, historical papers, and other 
unique or rare items on demand, collections once available only by satellite, video, or 
actual visit. 

To summarize, following Means and her colleagues (1993), we have argued that 
technology effectiveness should be defined in terms of the extent to which technologies and 
technology-enhanced programs support engaged learning and instructional reform. We then 
developed the idea of conceptualizing learning and technology as two intersecting continua 
(Table 1). Specifically, learning was conceptualized on a scale moving from passive to 
engaged. Technology performance was defined as moving from low to high. This 
intersection yielded four categories and four desirable trajectories. 

To elaborate on this framework, we enhanced the framework for learning developed by 
Means and her colleagues by adding one indicator category: the vision of learning, making a 
total of eight analytic categories, We also added indicators in each variable category to 
enhance the framework (Table 2). Similarly, we defined six neutral, analytic variables or 
categories for describing technology performance. Then we developed various indicators 
within each category (Table 3). This enhanced framework, we believe, could provide a 
powerful matrix for developing tools and for analyzing particular technologies and programs 
in broad terms. 

Local, State, and National Policy Issues 

In order for schools and others to design and use technology effectively to promote engaged 
learning for all students, certain elements inside and outside of the classroom must be in 
place. Specifically, we have identified five sets of policy issues related to learning and 
technology performance that greatly affect a school's ability to employ technology in 
classrooms for the kinds of engaged learning experiences identified by us and by Means and 
her colleagues. These policy issues are a critical component of our framework. 
Unfortunately, these issues are currently being addressed by different policymakers at local, 
state, and national levels and are therefore uncoordinated. More alarming, in m^ny cases 
these issues (e.g., the relationship between traditional distance learning providers and the 
Internet) are not being addressed deliberately and systemically by any group of policymakers. 

The first policy issue concerns equity, or the goal of universal participation. Although 
there has been a commitment to a national infrastructure and universal participation at the 
national level, this commitment must be made and, for the most part, funded at the state 
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level. And, as local control has often demonstrated, the final application of this ideal must 
be implemented at the local level and guided by local policymakers. Universal participation, 
as a policy goal, must mean that ever' student in every school will have access to and active 
involvement in an information highway that connects them with other students and to the 
world. 

This issue raises questions about funding for specific contexts (e.g., urban and rural); 
specific populations of users (e.g., poor and minority, children with special needs); and 
specific states (e.g., those economically depressed). Policymakers will be asked to make 
decisions about designing and financing state and multistate technology infrastructures that 
anticipate the high performance technologies we have described above. 

The equity issue also raises additional concerns about the host of internal problems that 
prevent schools from being able to participate, even if they have the technology. Some of 
the obstacles are a lack of (a) focus and time for quality, ongoing professional development 
for the technology training; (b) models for curriculum, instruction, and assessment that 
promote engaged learning and effective use of high performance technologies; and (c) school 
architecture that supports a community of learners, knowledge building, and multiple 
technology functions. This equity policy issue rests upon the belief that students now 
academically at risk can achieve to high standards and engage in challenging learning that 
uses high performance technologies; and therefore, they should have access to both engaged 
learning and high performance technology. 

Providing high quality technology access for all students to achieve high standards of 
academic excellence is the second policy issue. Whereas the first policy issue focuses on 
access, connectivity, and interconnectivity, this concern focuses on standards for learning as 
they apply to technology— ensuring that students at risk have opportunities to use the 
technologies to complete challenging tasks. This issue, as it applies to technology, calls for 
high standards for all children. This means making a commitment to establish high standards 
for technology access and use for all students through specific policies and financing 
strategies. The literature cited in the next section suggests that students academically at risk 
can benefit from learning paradigms that present challenging learning tasks and opportunities 
to use high performance technologies. Indeed, it may turn out that technology use to 
promote engaged learning is a very powerful veliicle for students at risk. 

Major barriers exist, however, to implementing such policies at the local level. They 
are: (1) local assessments that focus on low level and conventional objectives; (2) 
technology initiatives fragmented (separated) from curriculum, instruction, and assessment; 
and (3) tracking systems that separate students and technology into low- and high-level 
applications. What is needed is (1) permission for schools to experiment on using technology 
as a tool for restructuring classrooms and the schools themselves; and (2) integrating policies 
that relate curriculum, instruction, assessment, and technology so chat in practice schools 
curriculum, instruction, assessment, and technology are seamlessly integrated to support 
engaged learning. 
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Developing standards for technology performance and for the intersection of learning and 
technology so that all students have opportunities to reach the same high standards will be 
critical. A definition or' technology literacy is needed that requires teaching students t o use 
various technologies as tools to accomplish challenging tasks that empower their learning and 
are aligned to the workplace and community. This need has special implications for Goals 
2000, ESEA, School Improvement, and school-to-work legislation, as well as the design and 
funding for the national information infrastructure. We also need national standards for whai 
constitutes high performance tech ologies that promote learning. 

The third set of policy issues surrounds the coordination of technology choices and uses 
from K-12 to postsecondary and to work. The transition from school to work can be greatly 
strengthened by allowing students to become familiar with workplace technologies. 
Employability is an important concern for all students, and experience using technology with 
high transfer to the community and workplace is important. The present strategy for 
purchasing and using technology in K-12, postsecondary, and school-to-work programs is not 
coordinated and involves many different policy players and many different configurations of 
technology and telecommunications. Private sector and public sector planning and K-16 
planning could facilitate shared financing and improved articulation for school-to-work 
technology access and use. 

The fourth policy issue surrounds commitment to ongoing professional development 
that prepares educators to implement the instructional and curricular strategies implied in the 
enhanced framework developed by us. This commitment involves time, financing, staffing, 
and powerful models based on recent research on learning, professional development, and 
technology use emerging from the cognitive science and related fields. Moreover, such a 
conceptualization must be based on the assumption stated in Goals 2000 and ESEA legislation 
(in progress) that all students can learn to the same high standards, which has important 
implications for building planning and classroom management as well as seamless 
curriculum, instruction, and assessment. 

One final issue that must be noted for its absence in the literature on technology is the 
role of parents. While several programs involve parents or local community members, most 
do not. Not surprisingly, our experience in schools and SEAs suggests that many parents do 
not understand this major shift in technologies and programs— specifically, its importance for 
their children's experience in school and capability in the workplace. Historically, groups of 
parents across the nation have not supported initiatives focusing on teaching thinking. We 
believe that when this fear is coupled with technology, parents could feel very threatened if 
they are not brought into the partnerships (see description of Glenview in the Impact in 
Schools section following). 
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3. Learning, Educational Reform, and Technology 

In this section, we describe four changes that are redefining educational reform. First, the 
strong national movement to open up the Internet to schools has created significant resources 
for curriculum, instruction, and assessment models for schools as well as for professional 
development. Second, this new set of content and service providers will have a major effect 
on the technologies and programs used in schools. 

These forces may accelerate school restructuring considerably. Third, the nature of 
evidence needed to measure student achievement within this technology-enhanced 
environment will be significantly different from the focus on standardized tests used in 
traditional instruction. And, fourth, renewed interest and funding in school-to-work 
transition is linking workplace technologies with secondary and postsecondary educational 
experiences. 

The National Information Infrastructure 

The U.S. government has made a major commitment to develop the Internet as a globally 
networked technology infrastructure. The government is also committed to developing a 
National Information Infrastructure (Nil) as the human, policy, and fiscal infrastructure that 
guides the development of the Internet infrastructure (see Ramirez & Bell, 1994, for a 
comprehensive treatment of the Internet, the Nil, and education). The Internet was initially 
created to assist universities, government, and the military in communicating and 
collaborating. More recently, however, the developers of this infrastructure have set their 
sights on universal access. This includes not only opening up Internet resources to business 
and other adult users, but also to children in schools. We propose that the NH will fast 
become a profound force for ongoing professional development and systemic school 
restructuring, supported strongly by various agencies in the government including the 
National Science Foundation (NSF). NSF is issuing a major report on principles and goals 
of technology in education (Sabelli & Barrett, 1994) that will move the Nil beyond universal 
access to participation and connectivity. Several recommendations from that document are 
worth quoting: 

Technology freely used will change who is in control of information within schools and 
classrooms. 

Technology can be, must be, used as a tool for inclusion instead of exclusion. 

Technology policy must promote the integration of educational technologies in school 
with the technologies outside of school. (As a corollary, they suggest that we must 
eliminate policies that prevent the development of a reasonable technical infrastructure 
(i.e., an infrastructure that has distributed resources, not single source providers.) 
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Develop appropriate technology goals and change those regulations that are barriers to 
achieving those goals. 

The goals stated as "next steps" in this document are ambitious: First, define "full 
participation" by 1994 with expected outcomes* Second, provide electronic links to every 
school by 1996, every classroom by 1998, and networked student clusters within every 
classroom by 2000. The report urges that "the community at large must seek support from 
all sectors and design activities to implement these next steps." 

New Education and Network Service Providers 

According to the NSF report, one set of outcomes of this movement has to do with profound 
changes in what is delivered to schools and who is delivering it (i.e., who is in control of 
defining the content for students and professional development). We suspect that 
governmental and R&D agencies and electronic publishing will increasingly replace 
conventional textbook publishing as the next generation of content providers for schools. 

We believe that there are critical differences between traditional and next-generation 
approaches. Collaboration and worldwide networking with schools will become the norm, 
co-development will become commonplace, and shelf life will be measured in months rather 
than yeais as products are developed and refined using the Internet. 

Who are the new service providers? One category of providers is the array of 
government departments and agencies that are building the Internet and the Nil. For 
example, recent policies of the U.S. government have permitted the release of huge 
repositories of free information and educational resources to schools through the Internet. 
These resources include NASA, the U.S. Weather Service, federal energy laboratories, 
various departments within the government such as labor and commerce, and diverse 
oceanographic and environmental agencies. 

The activities offered by the providers in this category are quite varied. NASA, for 
example, has a model Classroom of the Future, massive databases of planetary images and 
other data available on the Internet, five regional teacher centers, curriculum activities for 
various projects, and opportunities for teachers to network with each other and NASA as a 
means of ongoing professional development. The JASON Foundation, supported by various 
government agencies, has similar activities. JASON V: Belize— Expedition Planet Earth, 
for example, which focuses on the rain forests in Belize in Central America, involved 
satellite-based delivery of on-line scientific activities, video teleconference to participating 
schools, bulletin boards for participants, curriculum materials, newsletters, student and 
teacher travel with the argonauts, and direct transmission of data sets to users. The federal 
energy laboratories also have diverse offerings to schools. In addition to releasing various 
data sets pertaining to the planets, weather, and energy, several of the energy laboratories 
(e.g., Brookhaven and Oak Ridge) are developing software to deliver 3-D images and virtual 
reality to schools using the Internet; many have school-based research projects; and some 
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have special services for teachers such as the Ask tiie Scientist program at Argonne 
Laboratory. 

A second category is agencies that serve schools in traditional ways. As a part of the 
government policies to develop the Nil and the Internet, various initiatives are focusing on 
redirecting these agencies to serve schools as content and service providers for the Internet. 
Specifically, this involves the services and products of the regional educational laboratories 
(RELs), museums, libraries, zoos, and various agencies in the areas of health. Indeed, there 
is a high level movement to encourage informal consortia among these groups. Both the 
Smithsonian Institute in Washington, D.C., and the Chicago Museum of Science and 
Industry, for example, have extensive school-based projects and curriculum materials and 
both are developing consortia to develop ongoing projects and outreach to schools using the 
Internet. The Chicago Library System (all the library groups in Chicago) is working with 
the University of Illinois Library to develop two-way video desktop conferencing among 
users. NCREL has a strong commitment to this movement and is moving forward to connect 
its seven states through intermediate units and school districts and schools to R&D resources 
that are multimedia and delivered over the Internet directly to schools. 

A third category of providers is the R&D community of universities and private nonprofit 
agencies devoted to improving education. We believe that schools will increasingly turn to 
the community as providers. We have identified several forraal and informal consortia who 
have much to offer schools in terms of content and services cased on recent research on 
learning. These groups also are very involved in shaping the policies, the national R&D 
agenda, and information highway system that will carry the content and services they have 
designed. 

• One constellation of high end technology and learning developers is in the Boston and 
Cambridge area: Bolt, Beranek, & Newman, a primary developer of the national testbed 
concept described below; TERC, which focuses on technology resources and projects in 
mathematics and science; the Massachusetts Institute of Technology, which has done 
work with LOGO in various schools; and the old Bank Street technology group now at 
Education Development Center (EDC). 

• Another constellation is the Cognition and Technology Group at Vanderbilt, which is 
developing an array of technologies and programs around the concept of instruction 
anchored in narrative formats and multimedia technology. This group is forming a 
consortium with the Bereiter and Scardamalia group at the Ontario Institute for Studies in 
Technology (OISE) working on CSILE (Computer Supported Intentional Learning 
Environments), and the Brown and Campione Group at Berkeley. 

• A number of universities also provide major resources and support services to teachers, 
schools, or museums for a specific project or a group of projects: the School of 
Education and Social Policy and the Institute for the Learning Sciences at Northwestern 
University; the Image Processing for Teachers project at the University of Arizona; the 
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Common Knowledge Project for the Pittsburgh Public Schools developed by the 
University of Pittsburgh; and the many universities with strong technology-enhanced 
curricula for preservice teachers and/or technology research agendas such as the 
University of Michigan, Indiana University, and Ohio State University, 

• The business community and nonprofit agencies are increasingly involved in providing 
schools with products and services that reflect recent research on learning and reform. 
These include curriculum materials developed at WASATCH, the national Apple network 
based in California, the Becoming a Problem Solver Series supported by SRI in San 
Diego, the Learning Circles initiative from AT&T, and the Buddy System at IBM, 

A fourth set of providers is the community of electronic publishers; broadcasters; 
distance learning providers (see Westrum for summary of the research base, 1994); the 
video/film industry; telecommunications and computer companies; and the business 
community at large (the giants and small businesses) who will provide content, networking, 
and educational services. These providers will have increasing interest and control in what is 
available to schools, homes, and the R&D community. 

What is to be provided? Networked schools can receive up-to-the-minute data from 
every sector of society around the globe, and they will be able to import a wraith of 
curriculum frameworks and materials to construct their own projects and curricula. They 
will receive ongoing professional development support based on research on learning. These 
schools can become part of a worldwide network of schools that collaborate with each other, 
research agencies, and practicing professionals to build knowledge communities. Many of 
the new data sets they will access will be in picture and video formats, A major part of this 
infrastructure is the development of digital libraries and museum learning environments that 
help students and teachers access, browse, manipulate, and interact with image and video 
data. Leaders of this movement to develop network and content services recognize the need 
to design new formats that avoid the linear and static data that often have been the norm for 
library and museum collections. 

Different Kinds of Evidence for Technology Effectiveness 

There is a growing skepticism about the use of standardized tests and traditional study 
designs such as pilot studies with control groups for measuring technology effectiveness. In 
place of these measures there is a growing focus on student performance on authentic tasks 
and projects in the context of real audiences. 

One important new concept for technology and program evaluation is the notion of 
national testbeds developed largely by BBN (Bolt, Beranek, & Newman, 1992, 1993), 
TERC, and others to study technologies and programs that move toward universal access and 
participation in mathematics and science. According to Hunter (1993), a testbed is a 
combination of organizations, telecommunications networks, and educational innovations that 
involve ongoing collaborative inquiry in networked communities over long periods of time, 
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In testbeds, teachers, students, scientists, educational researchers, and administrators work 
together to develop expertise and to evaluate the costs and benefits of a given technology or 
program (or a multiplicity of them) as well a* issues of scaling up to serve mass populations 
(See also, Bolt, Beranek & Newman, 1992). 

In testbeds, attention to issues of te:hnology effectiveness, cost effectiveness, and 
benefits for students and schools is ongoing through day-to-day communications and frequent 
interactive studies. Testbeds also involve an array of qualitative and quantitative measures. 
These measures include surveys of teachers and students, in-depth interviews, analyses of 
recorded communications and artifacts, and classroom observations. Student and teacher 
comparisons between testbed and nontestbed schools focus on documentation of changes in 
attitudes, beliefs, or behavior including use of tools and resources. Measures of student 
achievement make use of portfolios and project paths, contributions to the scientific or 
literary communities, and locally developed assessments. Standardized tests, such as the 
individual and group tests used in California, may also be used if they measure specific 
reform goals. The testbed analysis looks at changes in school organization, policy, 
programs, and practices. Currently, there is a wealth of formative data, but most testbeds 
are only just gathering or analyzing summative data so that they are unable to make strong 
recommendations about effects on student achievement at this time. 

A major issue of technology effectiveness for all service providers is the "scale up" 
challenge. Most service providers from whom we have collected information say they 
currently serve about 500 schools. All such providers are making plans for serving much 
larger numbers. NCREL— which has the largest educational laboratory region— serves over 
22 percent of the nation's children. Staff are considering ways to grapple with what we call 
the 20 percent problem— how to provide access to proven programs to 20 percent of the 
schools in the region. We think it is likely to include some combination of the Internet and 
next-generation distance education providers using satellite, c^ble, and other forms. 

Impact in Schools. Several references have been made to the impact of technology on 
student achievement and school restructuring. Again, we note that much of the data below 
are preliminary but they are encouraging: 

• Greatly expanded information exchange capabilities. This includes access to text, audio, 
and video, as well as search tools and bulletin boards for exchanging local and global 
resources. These capabilities include new technologies and tools such as World Wide 
Web, the Web Crawler, e-mail, gophers, distribution lists, and group mail reflectors. 

• New understanding about learning and understanding. As students and teachers 
increasingly use the very complex information sources and tools provided on the Internet 
and directly by research agencies, they are developing new skills, knowledge, interests, 
and dispositions. We are referring here, for example, to the array of browsing, inquiry, 
and navigational skills for nonlinear learning in contexts. Further, these new learning 
environments have few familiar structures and markers as well as involve asynchronous 
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communications, messy data, multiple perspectives and formats, distributed logic, 
entangled domains, and co-construction of knowledge or knowledge building. Such 
contexts stimulate new understandings of causation, intelligence and cognition, learning 
opportunities, learning environments, and achievement itself—for students and teachers. 

Curriculum organized as projects involving sustained and complex co-investigations. 
These projects exist across geographic and political boundaries and allow students to 
interact with practicing scientists and other professionals. Such projects actually offer 
students the opportunity to make contributions to science, literature, and other areas 
within local and global communities. Especially noteworthy are examples in the 
literature from TERC, BBN, and elsewhere in which academically at-risk students make 
significant contributions to the broader learning community, perhaps by creating 
multimedia and well-researched museum exhibits (e,g,, Collins, Hawkins, & Carver, 
1991; seu below also). 

Changes in student and teacher roles in the classroom. Teachers and students are seen 
as contributors to knowledge, able to take charge not only of learning but also of creating 
and directing learning opportunities, and as co-investigators and citizens of the global 
learning community. Teachers and librarians are also seen as managers or brokers of 
resources. In addition, technology specialists are taking on a variety of roles that include 
expertise in linking to the R&D community. 

Changes in the conceptualization and practice of professional development. Professional 
development is ongoing. It is delivered in diverse media including e-mail, telephone, 
facsimile, video, audio and video conferencing, computer-driven software, satellite-based 
programs, and over the Internet—in many cases in network mediated contexts. Even 
print materials are trying to look and behave more interactively with innovative formats, 
customized structures, more hands on activities, and multimedia linkages. It is also 
delivered face-to-face at national, regional, state, and local centers as well as in schools, 
universities, and various other agencies that traditionally have not provided professional 
development experiences such as NASA, the JASON Foundation, museums, and 
libraries. Location for professional development delivery is now anywhere in the world. 

The persons designing and providing the professional development experiences include 
some new experts in the "real-world" including: practicing scientists, artists, 
telecommunications \ mdors, end authors. Agencies such as TERC, BBN, Argonne, 
AT&T, the Learning and Technology Group at Vanderbilt, now send out teachers on 
loan from schools serving as consultants and leaders, network support staff, and 
technology specialists. The transmission model has been replaced by mediating, 
coaching, sharing, listening, supporting, co-constructing, co-designing, co-investigating, 
co-producing, and co-presenting— all using technology. The processes of professional 
development experiences involve cycles of discussion, observing, modeling, reflecting, 
designing, trying out, revising, and refinement of strategies and activities over the 
years— using technology for input and for building knowledge with others. The 
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traditional isolation of classroom teachers has been replaced by global learning 
communities and communities of practice, (e.g., Hunter, in press; Roupp, 1993). 

Teacher roles outside the classroom are changing, too as the result of technology. As 
indicated above, teachers are increasingly consultants and technology specialists; policy 
decision makers for technology purchase, design, and use in schools; curriculum and 
software developers using multimedia, interns at educational agencies using, librarians, 
and navigators through the Internet. More often, teachers and school administrators are 
also key links between using technology at school and using it at home and in the 
community. 

Accelerated curriculum and school restructuring to promote learning. It has been widely 
assumed in both the research literature and practice that restructuring takes time, and that 
five years is not unreasonable for schoolwide restructuring. Numerous sources in the 
literature refer to using technology as a major strategy for school restructuring and for 
galvanizing teachers to act as change agents within the school. The primary source of 
this energy is the extraordinary motivation that many users derive from working on 
authentic tasks and collaborating with a broader learning community. Many schools are 
implementing interdisciplinary curricula and themes that center around the use of 
technologies— especially those involved in accessing the Internet. Moreover, student 
projects focusing on challenging authentic tasks often bring curriculum, instruction, and 
assessment that are not only aligned but essentially inseparable and seamless. 

Thus, when technology has such a galvanizing effect, the time needed for curriculum and 
school restructuring may be reduced significantly. Goldberg and Fortunato (1994) 
estimate that these processes can be done in three years, starting from scratch (i.e., 
moving from a very limited emphasis and capability on technology to schoolwide use of 
technology to conduct projects). 

An example from Glenview Public Schools in Illinois will illustrate how fast this process 
can be when schools do not start from scratch. Essentially, there were two technology 
initiatives. First, there was a major movement within the school district and community 
to network various groups and agencies using broadband cable. While elements of this 
networking were already in place prior to 1992, and various community members knew 
about these elements, concerted action began in 1992 and was the system place in the 
spring of 1994. The communications, technology, and overall plan for this process is 
described in detail by Mundt (1994). 

During the school year 1992-1993, the district had separate initiatives for instruction and 
technology. For instruction, they were providing ongoing professional development for a 
thinking skills program that was to drive the agenda for curriculum, instruction, and 
professional development. In a separate but parallel initiative, the district had established 
a well-equipped computer lab in every school with enough computers for each student in 
the class to use a computer. 
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In the spring of 1993, the district hired a consulting group to evaluate the district's 
current use of technology and make recommendations. The group stated that the 
district's technology operation, organized in a single learning laboratory in each school, 
was impressive as a beginning. However, they maintained, if the teachers and students 
were to use technology as a tool, it must be accessible in every classroom. The group 
estimated costs to accomplish this. The recommendations were published in a letter 
mailed to every resident, and the district subsequently reallocated funds to support the 
recommendations over a 4- to 5-year period. 

As part of the plan, in 1993-94 the Technology Director, James Flanagan, began to seed 
high quality technology programs in classrooms throughout the district so that a few 
teachers were linked to stellar technology projects. In the fall of the second year 
(September 1994), the district was to be divided into pods or trees. One tree was 
technology so that all classrooms in that tree would have very high performance 
technologies and various technology-enhanced programs that would yield sustained and 
engaged learning. 

A major part of this plan was to distribute the research literature from each technology 
project and other technology research to teachers and district staff, involve them in 
powerful projects, such as Global Schoolhouse and CoVis, and provide ongoing staff 
development. These projects were so successful that by April 1994 the technology plan 
was the driver of curriculum and school organization at the district level with very high 
levels of enthusiasm from students and teachers, according to district staff. That is, the 
learning paradigms underlying these programs emphasizing challenging, authentic tasks 
and collaboration were so effective and attractive that these concepts began to extend and 
redirect curriculum, instruction, student and teacher roles, and assessment. Teachers 
then began integrating the thinking skills into the projects so that the technology initiative 
and the thinking skills program became integrated conceptually. Thus, the learning 
research reflected in the technology initiative became the driver for the instructional 
model for the district. 

Another example of very accelerated school restructuring concerns a massive effort to 
demonstrate the effects of technology on selected schools abroad in the Department of 
Defense (DOD) for a major international demonstration of networked technologies 
(Goldberg & Fortunato, 1994). The initiative involved radically restructuring these DOD 
schools for large scale, collaborative projects. One of the projects involved doing 
something that had never been done before: creating an original orchestral arrangement 
using the Internet such that the different instruments would be played in different 
locations around the globe. Overcoming many technical obstacles, resistance to school 
restructuring, and the constraints of working at a distance through technology, the project 
actually failed in two attempts to rehearse the piece of music so that when it was played 
for the Vice President, no one was certain it would work. However, after some delay, it 
did. 
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• Changes in student achievement. There are a number of technology-enhanced curricula 
that have well-documented results in terms of improving student achievement even using 
standardized measures. Pogrow's (1990) HOTS Program and The Jasper Woodbury 
Series, developed by the Cognition and Technology Group at Vanderbilt (1992b), for 
example, are perhaps the most well-documented and both are effective with students at 
risk. Numerous other programs have very promising data from project documentation, 
surveys, and classroom observations regarding students at risk. Other studies contain 
many references to improved understanding of concepts, more engagement in active 
learning, preference for more difficult questions and challenging tasks, more student 
leadership, and more engagement in authentic tasks that provide real products, services, 
and outcomes for actual audiences. Especially rich illustrations come from the TERC 
testbed (e.g., Weir, 1993). 

Some of the stories/cases from this literature are very dramatic. In one study, learning 
disabled students were required to use integrated multimedia technologies to make 
presentations. Teachers who initially did not believe that students could complete the 
task were awed and personally moved at the quality of the presentations; they realized 
that these students were able to tackle considerably more difficult tasks than previously 
assumed (Cognition and Technology Group at Vanderbilt, 1993; Hasselbring, et al. , 
1994). In another example, low-achieving students in Rochester developed an exhibit on 
the city that was displayed in the local museum (Collins, Hawkins, & Carver, 1991). 

Another example comes from Whittier Elementary School in Chicago. Working with 
NCREL to implement instruction based on authentic tasks, the school took a considerable 
risk to abandon a traditional summer school curriculum and implement one based on 
authentic tasks including using technology and desktop publishing to create and publish a 
school newspaper. Teachers who had been highly skeptical all agreed that it was the 
most successful summer school in the history of that school and was instrumental in 
restructuring the regular curriculum during the school year. Since then, the school has 
rapidly implemented networked technologies and projects, including several international 
ones. 

Two final examples of student achievement illustrate what can happen when students 
engage in authentic tasks that actually change their roles and relationships. One success 
story is taken from the Image Processing for Teachers project at the University of 
Arizona. This project provides students at various grade levels with large sets of 
planetary and weather images from NASA and other sources as well as actual (messy) 
data such as x-rays and archeology findings from various research laboratories. The 
students' role is to clarify and analyze this data, using an array of image processing tools 
from NIH Imaging. (This is the same software from the Internet used by many scientists 
working with images.) 

During this process, students and teachers regularly communicate with project staff who 
are trained content specialists. In the process of working with these images, students and 
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teachers can, for example, predict with 80 per cent accuracy certain lung diseases, piece 
together fragments of ancient pottery, conduct forensic research, generate new image 
representations from a given image including three dimensional ones, clarify photographs 
full of static or discoloration. Moreover, project staff have worked to provide 
historically interesting data such as actual images from photographs taken from space, 
slides of tissue used in research on the effects of alcohol on liver, and images of the 
brain in research just being conducted on the effects of drugs. This project has spread 
rapidly across the nation, sometimes involving a whole school after only one teacher is 
trained and can be done with one computer per classroom. There are many stories of 
high-level achievement among students at risk and their teachers including presentations 
of findings at professional conferences. 

The second example involves middle grade students at Pease Elementary School in San 
Antonio, Texas, who were taking air samples using an air pump designed by TERC 
(Berenfeld, 1993). Because the students were not permitted to go out into the 
environment, they measured the air in the classroom, found too much C0 2 , and 
concluded that it was an air pollution problem. The school called the local 
Environmental Protection Agency. Officials visited to measure the quality of air at the 
school using equipment very similar to that used in the classroom and confirmed the 
students 1 measurements. Both sets of findings were communicated on the Internet. 
Students from a school in South Carolina commented that they had compared air samples 
in regular and mobile classrooms, expecting the latter to be worse, but found the 
opposite. Apparently the opening of doors in the mobile units brought fresh air into 
those classrooms. According to Pea (1992), "the pedagogical goal is to have students 
better able to engage in appropriate conversations about the conceptual content they are 
investigating. H As part of the conversations among the participating schools and 
practicing scientists, the students were able to reason that the problem was ventilation, 
not pollution. More important, their conversations with local officials and with scientists 
gave them new roles and ways of thinking about themselves as learners and researchers. 

• Changes in the definition of learning community. We used to think of school as a place 
where students learn. Then we understood that schools and local communities had to 
function together to help students learn, so that there was a broader learning community. 
Now we think of national school networks connected electronically such as those 
sponsored by BBN (1992) and the Cognition and Technology Group at Vanderbilt (in 
press) as well as global ones such as the Global Lab international network at TERC 
(Berenfeld, 1993). 

All this is not to say that implementing technology is easy. Certainly, it has been clear 
throughout this paper that too many instances of technology implementation are ineffective 
because the technologies and programs did not promote engaged learning and/or did not 
provide ongoing professional development. However, there are other problems such as the 
cost of technology; lack of information about where to go to find effective help in purchasing 
and implementing technology; shortage of experienced professional development staff; and 
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barriers within schools such as resistance to change, fear of technology, and lack of time and 
funds for teachers to develop new skills (e.g., Office of Technology Assessment, 1993; 
1989; 1988). The reality is that many teachers and school administrators have never heard 
of the Internet and have little to do with technology. 

Technology and School-to- Work Initiatives 

The literature provides a number of options on how technology can help align student skills 
with those needed in the broader community and workplace. Technology may be used to 
support most of the basic workplace skills, such as negotiation, collaboration, and knowledge 
of systems. By extending our efforts, we could provide students with many of the basic 
workplace technologies such as word processing, use of multimedia formats for 
presentations, and spreadsheets. Of more serious concern is how to give students access to 
some of the very expensive workplace technologies that are context-specific, such as imaging 
technologies (e.g., CAT scans) used in the medical and biological sciences and large printing 
presses. Imaging will be increasingly important to all citizens in the 21st century because of 
its superior capability to communicate information, especially technical information. The 
development of imaging technology devices and technologies such as photography, motion 
pictures and video, television, and computer workstations has facilitated the production and 
communication of knowledge. Images have become essential to science, medicine, industry, 
education, psychology, and culture generally (Beck, 1992, 1994). It is vital for students, 
teachers, schools, and community members to understand the basic principles of this 
technology and the scope of its uses in various fields such as radiology. 

Some mergers of school-to-work transition initiatives would promote the use of 
workplace technologies by students. For example, museum exhibits and specially designed 
work stations for schools allow students to have sustained learning experiences with high 
performance technologies that promote engaged learning. We also found two emerging 
models of teaching and learning that support quality learning experiences and address the 
new focus on school-to-work transition: (1) cognitive apprenticeship and (2) the idea of 
knowledge-building communities. To explain: These models emerge from research on 
learning, but they have a natural conceptual affinity for school-to-work issues because both 
seek to engage learners and communities in social relationships that are so critical in the 
workplace. 

Cognitive apprenticeship, for example, is built on the (occupational) apprenticeship idea 
that uses the teacher as coach and mentor. Thus, the elementary or high school student in 
the role of cognitive apprentice is able to have a relationship with his/her teacher as coach 
and mentor that is metaphorically like the occupational apprenticeship. The knowledge- 
building model in eduction was designed in part to simulate how members of the learning 
community each provide multiple perspectives on a problem— in the world of work, in 
community problem solving, and in R&D collaboratives. 
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4. Regionally as a Strategy for Designing and Implementing an 
Effective Technology/Learning Interface 

Universal Access and Technology Growth as Problems for Education 

There is a need to develop both a human and technology infrastructure that provides content 
and services to K-12 education. Decisions with profound consequences are being made about 
how various technologies will be integrated into the home, business, and entertainment 
markets within and across the United States. Education as a public institution needs to be at 
the table with business, the telecommunications industry, the power companies, the military, 
and others when vital issues of access to information are decided. Many of these industries, 
such as telecommunications and power, work on a regional service base* We propose that it 
may be to the advantage of education to explore multistate cooperation to take full advantage 
of vendor regional delivery capacity, economies of scale, and shared planning and 
implementation resources. 

Technologies are expected to reduce greatly inequities in many contexts because in theory 
they can provide universal access to information. That is, since everyone should have equal 
access, everyone should have equal opportunity to learn. As noted above, the government 
and others are moving from the concept of universal access to the assumption of universal 
participation. However, too many poor schools face obstacles that hinder and/or preclude 
their access to these learning opportunities. These obstacles include (1) lack of funds to buy 
the needed technology, (2) curricula and assessment programs that focus on low level skills 
even when technology is applied, (3) school faculty who r.ieed to develop the instructional 
strategies to use the information that they can access through the technology, and (4) 
bureaucracies that keep communication and development from moving beyond the walls of 
the school into business and community sectors. 

Goals 2000 and the technology legislation in progress may alleviate some of the funding 
crisis and go some distance toward assisting poor schools in providing ongoing professional 
development. Nevertheless, it is quite possible that richer schools, which are abke to access 
and use information and research resources, will get "information richer* 1 while poor schools, 
by comparison, will become significantly " information poorer. 11 As the richer schools 
develop greater expertise, they also develop a much higher capacity to develop themselves 
and their related agencies. Poor schools are often preoccupied with curricula addressed to 
low level standards and objectives. When more schools are given permission and incentives 
to develop innovative curricula aimed at high standards, it will be much easier for them to 
use their funds to purchase technologies configured to support new learning paradigms. 

Four related problems also exist. First, many researchers have cited the limitations of 
the factory model of schooling in curriculum, instruction, and assessment. What is 
mentioned less often— if at all—is that top-down management and high volume have been the 
economy of public education as an institution. This also happens to be true in many 
businesses, according to Reich (1991). Textbooks and equipment, supplies, and consulting, 

46 



ERLC 



52 



information, and management services all are purchased and distributed according to 
hierarchical, centralized structures— structures that are out of alignment with today's needs in 
both education and business. The new technology-driven organization and delivery of 
information, services, and equipment can and must address this issue. 

Second, systemic reform in the cities has been long challenged by school funding 
formulae that depend on residential property taxes. If education is to change, the tax and 
funding structures of schooling must be part of that change. Ironically, the funding crisis in 
public education has given birth to new public/private sector ventures and some of these 
ventures are bringing new funding for technological initiatives. Following are three 
examples: (1) the plan for the Edison Project, which will provide and support high 
performance technologies that are oriented to sustained learning and management (Edison, 
1993); (2) the Television Curriculum Network described in Newsweek (Toch, 1994, p. 69) 
developed by Eric Jones who was instrumental in developing Channel One; and (3) Whittle's 
Channel One, which delivers news supported by advertisements from commercial vendors. 
Television Curriculum Network— unlike Channel One, which does not integrate the brief 
video news modules into the curriculum, will provide video segments that are linked to 
various standards and curricular objectives and so will be integrated. 

Third, the scale up challenge will probably require the formation of new organizations 
and consortia. It seems unlikely that any one institution or agency will be able to provide 
very large-scale technology services to schools and still maintain quality services. In part, 
this is because no one technology should dominate large efforts, and because large efforts 
likely will combine technologies— such as linking telephones and televisions, which many 
schools have— with computer-based approaches and technologies from industry. We believe 
that some creative approaches to conceptualizing the scaling up problem are required. An 
approach to this problem is discussed below. 

Fourth, there are broader economic reasons to think about strategies that go beyond a 
given city or state. Ramirez and Bell (1994) for example, have argued that the purchasing 
power of a multistate agency is far greater than that of a single state. Ohio State Senator 
Horn (1993) has argued that regional university consortia would provide the academic 
resources needed to develop the technology expertise to support the aerospace industry. He 
would also join industries in such a region to commercialize the aerospace technology for use 
in the private sector, and Gooler (1994; NCREL Policy Brief on Technology Infrastructure, 
1994) has argued that multistate or regional consortia would be powerful enough 
economically and politically to bring education to the policymaking table for decisions about 
technology and telecommunications. 

There are many who think (1) that the crisis in education is urgent especially regarding 
issues of equity, technology, and the skills needed for the 21st century; (2) that there are 
important opportunities to restructure schools with major initiatives from the government in 
Goals 2000 and ESEA legislation, the National Science Foundation, and other agencies; and 
(3) that the window to address these issues will fast close as massive human and technology 
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infrastructures are put in place without due regard for (e.g., National Center for Education 
and the Economy, 1990; Reich, 1992; Triangle Coalition for Science and Technology 
Education, 1989). 

A Proposed Approach 

The issues outlined above could become the incentives for coalitions and consortia to form 
regional initiatives The challenge is to think in terms larger than a school, district, or state, 
thereby sharing the cost of the human and technology infrastructure. Such an infrastructure 
is crucial to changing the opportunities to learn for millions of teachers, students, and 
administrators. Such an infrastructure also would give education the strength that it needs to 
maintain a place at the technology table on behalf of school children. Educators need to 
consider carefully the consequences of not having the political will to shape and manage 
technology opportunities as other organizations and groups are doing. Many key players are 
already organized in terms of regions that are fairly congruent. For example, there is 
considerable overlap in the Midwest among the following regionally-based institutions: 
Ameritech; Argonne National Laboratories; Central Education Network, a PBS affiliate; the 
Council of Great Lakes Governors; the Great Lakes Collaboration; Midwest Policy Institute; 
the technology network of the Committee on Institutional Cooperation (CIC) supported by the 
Big 10 and other universities; and NCREL. Forming interstate, regional and educational 
consortia would escalate many of those relationships significantly to create mutually 
beneficial enterprises. 

In taking the technology challenge to a different level, old rules and relationship* do not 
dominate; everyone is a learner, and a pioneer exploring new frontiers. For example, the 
cost of developing large repositories of digitized information resources is huge. Education, 
business, the military, and energy groups obviously can accomplish more on this task 
together than in isolation because they can work more economically and efficiently. If the 
appropriate players are at the table from the outset, school accessible information bases can 
be designed for multistate use, with each institution and agency contributing its particular 
core competencies for development. 

The question has been asked, Why can't we just depend on the Internet? Unfortunately, 
the Internet does not yet have the capacity to deliver the full use of video and images 
effectively. Increasingly, these modalities will be the currency of information trading. 
Imaging is required to access much of the most interesting data from NASA and the U.S. 
Weather Service, museums, public libraries, and historical document collections— not to 
mention all of the fields that are essentially visual in nature, such as the medical sciences, 
advertising, and journalism. Nor can the Internet effectively deliver two-way interactive 
audio- and videoconferencing (other than desktop capabilities). Obviously, these technologies 
require powerful national and regional servers, ways for schools to share the costs of last- 
mile wiring for high speed telecommunications, and delivery systems for home 
communication with schools and other educational resources—all of which are very 
expensive. 
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A further problem is the interface between providers, the Internet, and schools. The 
providers provide content and some services such as curriculum development, ongoing 
professional development activities, and some technology training and support. The Internet 
is the vehicle which providers use to transmit all text-based and some image-based data and 
services. But many schools are really at a loss to use these data and services optimally. 
This is in part because they spend their energy in other directions, in part because they are 
not well-linked to the data and services, and in part because many, if not most, schools do 
not know how to use these data and services to restructure schools and schooling to promote 
engaged learning. 

The point here is twofold. First, schools, not just teachers and administrators, need 
ongoing help in developing effective technology plans; in accessing the most recent research 
on learning; in restructuring their curricular frameworks schoolvdde to focus on and integrate 
this wealth of free data and services; in funding strategies; and in coordinating school 
restructuring in school improvement plans. Second, the Internet and the emerging video and 
imaging technologies could be used to change the economic basis of schooling by changing 
how information and services are delivered to schools across the nation. That is, the content 
and services that are available through the Internet and other telecommunications— many of 
which are free or low cost— could ultimately replace most of the textbooks and other costly 
instructional materials, software, and programs that currently devour up most school budgets. 
If each school were to draw largely upon these free resources and services, schools could 
spend far more of their resources for additional staffing, local curriculum development, 
technology growth, development of technology staffing, ongoing local staff development, and 
school restructuring. Moreover, if schools all across the country were to do this, the 
economy of schooling would be radically altered. 

How does all this relate to the question, Why can't we just use the Internet? The bottom 
line is that such massive and radical changes go well beyond the scope of tte Internet or 
indeed any one agency. What is needed is a set of regionally-based agencies to coordinate 
the evolution/natural emergence of such trends— or to drive these trends as their primary 
agenda. Such agencies would have to be well-equipped, well-planned, and well-coordinated 
not just to deliver school restructuring services to schools to coordinate technology and 
learning but also to plan and monitor the massive economic changes such a redistribution 
could yield. Moreover, such agencies could have as part of their mission the need to address 
the many equity issues relating to technology and learning— especially providing help to poor 
rural and urban schools. Needless to say, there are in-depth policy issues inherent in such a 
bold initiative. 

Scaling Up and Moving Out 

What follows is an outline of some innovative regional agencies that could be critical to 
address issues of equity and economy. Specifically, we develop the idea of a regional 
information port, a regional school service cooperative, regional service universities, and 
educational enterprise communities that would form new kind of testbed, a testbed that 
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focuses on region as the unit of change and deals with many of the political changes that are 
needed to support sustained change within a region. 

Regional Info-Port. One strategy might be to create regional information distribution 
and coordination centers that would involve a diversity of players using the Internet and other 
free telecommunications. The Port would (1) transport low-cost or free resources for 
schools; (2) support school collaborations using video, audio, and text technologies, and 
focus on bringing the poorest schools in urban and rural contexts into collaborations; (3) link 
the schools to practicing scientists and community members across the globe— a task that will 
soon need much greater coordination than is now the case; (4) work with schools to develop 
technology plans, and work with higher education and other agencies to develop training and 
professional development programs for technology specialists and librarians; and (5) provide 
ongoing support for school restructuring to promote engaged learning— again especially for 
schools for the poor and minorities. 

Each Port could be developed as an enterprise web (Reich, 1991), This would involve 
convening appropriate persons or agencies who are expert in all the areas necessary for 
successful adaptations and who would customize services for specific customers. Regional 
educational laboratories (RELs) could take the lead in convening the providers, the 
technology infrastructure agencies, and the schools. Each REL could convene the regional 
federal energy laboratory; representatives from museums, library systems, government 
agencies, universities, and other major providers; telecommunications providers; and various 
representatives of schools, intermediary units, and state education agencies. 

Content providers might include libraries, museums, the U.S. Weather Service, 
oceanographic and anthropological researchers, disease control and other medical centers 
with public databases, and NASA. These content providers would be added to those directly 
servicing public education, such as the Department of Education, national R&D centers, 
technical assistance centers, RELs, universities, SEAs, and standard-setting agencies such as 
the National Board for Professional Teaching Standards and intermediary units agencies that 
are designed, to support school and state education agency initiatives. 

Regional Info-Ports would create a distributed technology infrastructure that could serve 
thousands of users simultaneously with quality resolution and access. The human 
infrastructure needed to develop this technology and to share in the costs could include a 
configuration of any of the following: computer companies, the Departments of Labor and 
Energy, telecommunication agencies, the power companies, local private sector groups, the 
military, technology support systems, and local civic organizations such as chambers of 
commerce and individual Rotary Clubs, in collaboration with the educational agencies 
described above. 

Regional School Service Cooperative. In addition to an information port, there would 
be a need for service outreach— regional school service cooperatives. The goals of the 
service cooperative might include: 
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1. Helping schools access and use the resources available from the Internet and the Port in 
ways that both address national initiatives such as Goals 2000 and promote collaboration. 

2. Promoting equitable access to and use of technology-enhanced learning opportunities for 
students, community members, and teachers. 

3. Developing a new generation of regionally based and supported learning communities 
that define the learning place as wherever the learner can reach technologically: 
"virtual" and real learning places that would be open for use around the clock to serve 
various learning needs, and learning places in which the participants are both contributors 
and consumers of reserjrch, products, and services. 

4. Developing evaluation designs for the use of technology and assistance in implementing 
those designs. 

5. Studying and developing new policies to cover the new technology-supported learning 
contexts and situations, including policies and/or scenarios pertaining to the economics 
using Internet resources in place of or as a supplement to textbooks* 

6. Providing training and support services to use technology and to develop plans to 
"grow" technology in schools so that teachers and students can use it as a tool to solve 
problems and address needs. 

7. Working with schools to generate learning experiences for professional development 
including the development of roles such as digital librarian, project manager, computer 
specialist, and staff developer in an age of global technology. 

The following are some of the human resource questions that will emerge and will need 
to be addressed by R&D: What are the qualities of successful technology specialists 
necessary to support school development in techno A gy (a category not fundable in many 
schools)? How can they be trained so they can help schools make good decisions about 
learning, reform, and technology performance? Who should certify them, and who should 
train them? What is the role of a digital librarian? How will school-based projects be 
organized and indexed in libraries so that others in the school and learning community have 
friendly access to them. What are the standards of excellence in schooling for next 
generation professional staff, schools, and school systems? What are the characteristics of 
successful schools in a regionally based, global learning community? 

To reach the cooperative's goals, it will be important to develop delivery systems that 
facilitate collaboration (problem solving, inquiry research, consulting, mentoring, technical 
assistance, and other services delivered electronically). But schools also will need to develop 
new learning outcomes and models that help transform rich, electronically accessed resources 
into usable and powerful formats for learning and teaching. Means will have to be found to 
organize and catalogue not only external resources and services, so that schools can use them 
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better, but also resources, products, and services developed by schools. Indeed, clusters of 
schools might function as satellite enterprise webs, taking leadership in recruiting and 
mentoring novice schools. It also will be important to give participating schools latitude to 
reinvent schools using technology while still holding them accountable for standards of equity 
and excellence. 

Regional Service Universities. A regional service university might be an actual physical 
entity set up by a consortia of universities in the region that had its own human, fiscal, and 
political infrastructure. Or such a university could be virtual in that it could be a set of 
courses operating as an "invisible college" without separate human, fiscal, and political 
infrastructures. The purpose of a regional service university it to provide services to schools 
and school networks subscribing to mutual programs and goals. Courses available from 
contributing members might emphasize using research and technology for systemic, long- 
term school restructuring. Distinctions among teacher educators and university students, 
school teachers, and students would become blurred as teacher educators provide services 
and co-development processes, while school teachers and administrators work as co- 
developers and contributors to the school development process. These regional universities 
might develop programs of study and certification for new roles for schools such as digital 
librarian and school technology specialist (Jones, 1994). At the moment, most colleges and 
universities offer certification for teachers, administrators, librarians (not trained in the new 
library technologies), and media specialists. 

Educational Enterprise Communities. Increasingly, we are looking beyond individual 
schools to schools and communities as units of change. Several initiatives already exist as 
models. First, consider the city as the unit of change. The charter school movement, which 
concentrates on independently defined schools and school clusters, gives participants political 
permission within defined accountability structures to explore and develop new roles, rules, 
and relationships (NCREL Policy Brief, 1993). Municipalities as units of change— a concept 
developed in England by an Industrial Trust (see Abbott, 1988, 1990, in press)— also hold 
promise. Municipal efforts toward school and community learning can be seen in our 
country in programs such as Baltimore Reads, a public/private nonprofit initiated by the 
Mayor's office to engage institutions across the city in literacy so that schools, churches, 
business, libraries, and others would all be working to improve literacy skills. 

Second, we also must look beyond cities and states to regions. The Learning Zone 
legislation being developed in our country seems particularly promising (Public Act 88-200, 
Illinois H.B. 2282). There is recent legislation, for example, from the Governor's Office in 
Illinois to grant 10 percent of the schools in Chicago many of the rights of Charter Schools. 
While Learning Zones are not quite as independent as Charter Schools, the involvement of 
10 percent of the schools has the potential for a consortia that forms a networked system of 
education within the city. Such a zone could be strengthened dramatically through 
technology applications. 



52 



ERIC 



58 



The Co-NECT Network is also a powerful prototype. As a national network, it already 
is exploring the uses of technology in ways suggested here* More specifically, the Co-NECT 
School Partnership is a coalition of American educators, corporations, and community 
leaders which began in the Commonwealth of Massachusetts, but has now expanded to other 
states. Led by Bolt, Beranek, and Newman, this Partnership won one of the New American 
Schools Development Corporation awards. It proposes a design that enables local 
communities to create their own break-the-mold schools in radically new arrangements and 
connections. Focused specifically on inner-city schools, the Partnership works with each 
community to develop a design that will work in any school using a concept driven by five 
components: (1) project-based curriculum founded on challenging, in-depth seminars; (2) 
performance-based assessment founded on a set of challenging standards; (3) school 
organization involving multi-age clusters that are self-managing; (4) a restructured school 
community, featuring self-managing clusters of students, teachers, administrators, and 
community members; and (5) a flexible and open computer-based communications network 
that supports the project-based curriculum and the restructured school community and links 
them to a rich array of local, national, and global resources (see Bolt, Beranek, & Newman, 
1993). 

Start-up funds and special status provided to small pilot groups of schools and local 
communities could permit them to explore the day-to-day realities and policy issues involved 
in constructing technology systems that serve effective learning. We envision Educational 
Enterprise Communities spreading out over a multistate area with schools and communities 
working together with teams to identify needs, design and create learning environments that 
address those needs, and develop policies through rapid prototyping. Technology itself, 
electronic connections, will encourage the enterprises to keep learning from each other. 

Consider the opportunities that might result from a Regional Info-Port, Service 
Cooperatives, and Enterprise Zones linking various sectors of society and creating a regional 
community of learners: 

• Schools, communities, and researchers working collaboratively to develop local and 
electronic communities together would restructure education, altering what is defined as 
education, what educational materials are, how they are delivered, where education is 
received, who uses educational resources, and what constitutes literacy, especially 
technological literacy. 

• Publishers, researchers, universities, and others would deliver multimedia units of 
curriculum, instruction, and assessment directly to schools through Info-ports. Students 
and educators might take courses prepared or given live in other states, from other 
schools, or from the local district office. Teachers in different locations— for example, 
urban centers—could decide to collaborate to develop new curricula and opportunities to 
learn. The Algebra Project, for example, designed a transition curriculum for 6th, 7th, 
and 8th grades to bridge the conceptual gap many i students experience between arithmetic 
and algebra, and to prepare students to enter college preparatory math in high school. 
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• Educators within a region could access regional banks with (1) multimedia prototypes for 
curriculum, instruction, and assessment units developed by standards boards, state 
education agencies, districts, regional laboratories, R&D centers, and universities; (2) 
curriculum frameworks, with shells and frames for local curriculum development, and an 
array of locally developed units of instruction contributed by experienced groups; (3) 
library materials on topics relevant to restructuring to promote learning; and (4) libraries 
of videos and CD ROMS with master teachers demonstrating particular instructional 
strategies. 

• In schools with uplink capabilities for two-way video communication or with dishes able 
to receive programming, teachers could talk live to researchers and other teachers; watch 
a demonstration; present a demonstration for feedback; discuss diagnosing student 
problems; develop or co-develop integrated, multi-media materials with otter teachers; 
exchange ideas on specific topics; develop video conferences; extend their own video 
libraries locally by downloading materials from the bank and by creating new videos of 
the best teachers locally; and participate in video clubs to discuss one another's work as 
professionals. Teachers also could develop demonstrations of their teaching for official 
critique and evaluation for professional certification. 

• Participating schools might offer an array of services to community members from adult 
education courses to community outreach programs, renting out equipment and providing 
other school-generated services. 

Info-ports, service cooperatives, and regional service universities are about restructuring 
access to quality information resources and collaboratively redefining education as a public 
institution. They are about stepping out of the roles, rules, and relationships that constrain, 
shifting the focus to community-building issues and rejecting the status quo in education. 
They are about finding common ground among diverse stakeholders, sharing human and 
other resources, unleashing energy to imagine and create, and about learning how to 
cooperate and build consensus to solve educational problems together. They are about 
building enterprise webs that serve to build new economic and political bases for education. 



5, Next Steps for Research and Education 
Need for a National Database 

We believe there is a growing consensus about the good fit of the dimensions we and Means 
and her colleagues identified as sound systemic reform. At the same time, new technologies 
and studies of technology are emerging rapidly, constantly changing the outer limits of what 
is possible, what is now within reach. It is making less sense to rely on print medium to 
report these developments. Means, et al., do not cover many of the studies now available or 
emerging such as the Imaging Processing for Teachers project at the University of Arizona; 
CoVis at Northwestern University; NASA's newest Classroom of the Future; the network of 
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schools supported by Goddard; the Co-NECT schools; many of the tools being developed at 
the federal energy laboratories, universities, museums, and libraries; and some of the newer 
software such as Mosaic developed by the University of Illinois-Champaign-Urbana, and 
Minuet developed by the University of Minnesota. What is needed is a national database and 
communication system that will provide both high-level, synthesized research in casy-to-rcad 
formats for policymakers at all levels, on one hand, and an in-depth database with rich 
details in various categories for researchers and educators, on the other. 

Other questions and issues to include in a national database are: First, there is no 
common language to describe various ways "to grow" technology configurations in schools 
or districts. It would be helpful to apply Table 1 to specific learning and technology settings 
to get actual examples in each quadrant as well as to study how schools and districts move 
from one quadrant or paradigm to another. Second, many researchers who are developing 
technology programs have generated design features that should be examined systematically 
and comparatively. Third, analyzing the advantages and disadvantages of different 
implementation strategies could add a great deal to this literature. Schools and policymakers 
need such information to make decisions about implementing technology and learning goals. 

If we had a national database, it would provide needed information, a common language, 
and a common location for the various pieces that support implementation. Policymakers, 
institutions of higher education, and educators alike would then have access to information 
that could answer such important questions as: 

• How do successful schools "grow" technology expertise to address diverse technologies 
in the classroom as well as automated library systems? 

• How do such schools expand the role of librarians and others in order to organize and 
disseminate the array of new curricular modules and information emerging from the 
schools* participation in variqus technology projects, including input from other libraries 
and museums? 

• What kind of staff development principles and technologies are needed to support all of 
these evolving roles as well as the management of school operations, assessment, and 
finance? 

• What architectures will best support learning, communication, and collaboration as well 
as provide equitable access to information for all students? 

• What strategies do poor schools use to obtain funds for powerful technologies? Can 
there be a difference between the technologies of poor schools and more wealthy ones? 

• What motivates people to persevere in establishing technology-enhanced schools? What 
methods effectively overcome barriers? 
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• What strategies have been successful in spreading an effective technology program from 
a few sites to many? 

• What supports are required for successes in technology to be sustained, replicated, and 
combined with other technologies in order to become more powerful? 

• How will the different video-based distance education technologies (such as broadcasts 
and cable) interface with the development of instructional programs available through the 
Internet? Will they be integrated? What new capabilities emerge with such an 
integration? 

• Given how widespread VCRs and televisions are in schools and homes (Office of 
Technology Assistance, 1993), how can we stimulate or restructure the marketplace to 
provide incentives to develop more interactive television technologies to build on the 
existing video/television equipment in schools and homes? 

• How can education benefit from the application of entertainment technologies such as 
Nintendo? 

The focus on a national infrastructure should not suggest that we have all the needed 
research. Clearly, we are just at the beginning of a new era in learning and technology that 
is rapidly expanding and will be important to continue to study in the areas listed above, and 
others that will emerge. 

Need for Tools 

The framework developed in this paper provides a heuristic for technology learning 
evaluation. Ultimately, we will need tools to evaluate technology and learning that make it 
clear what a school's current realities are and compare this to its future vision. It must also 
be possible to compare one particular technology-enhanced program or curricula to another. 
Finally, it will be important to assist schools with inventories that help them identify 
preferable technology features and agencies that can support them. We have attempted to 
take a first step in this direction with the instruments in the Appendix. 

Additionally, there is a need to document (a) what tools already exist for the various 
teaching and administrative functions such as curriculum development, management of 
records, and professional development; (b) what tools are available that help students access 
powerful databases, make decisions, solve problems, communicate, and (c) what tools are 
needed for these functions and for learning? These two agendas— the need for a national 
database on technology and engaged learning and the need for powerful tools for education 
and for those who support them— are vital to technology policy and to improving the use of 
technology in schools to promote engaged learning. 
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6. Recommendations for Policymakers and Educators 
Implications of the Framework 

As indicated above, some recommendations regarding technology policy for schools and 
other contexts emerge from Table 1 and considerations of regional infrastructures. This table 
would clearly argue against strong support of a technology design that does not empower 
learning, regardless of whether or not it is costly, and regardless of the dominant technology 
or medium. This does not mean abandoning technologies presently being used for low level 
learning goals, especially if they deliver instruction to those who would otherwise not have 
access to it, or provide access to information that would otherwise be unavailable. What is 
important, for example, in distance education and ILS, is adapting these technologies to 
support higher learning and educational reform goals so that students in remote and special 
locations can access programs that braid engaged learning with high performance technology. 
It is also very important to support movement toward distributed networks as opposed to 
central source providers to build communities of learners that include students and teachers 
as contributors. 

This does not mean that all projects or services involving central source providers or 
software that is not networked should be abandoned. As noted above, there are some stellar 
projects, such as JASON, some of the NASA activities, Ask the Scientist, digital libraries, 
news and periodical services and others described above, that are essentially central source 
providers, but they provide a high quality product and service for schools that are highly 
motivating to teachers and students and promote engaged learning. The same is true for 
many tools that may or may not have open architecture and some of the high technology 
features, but which provide powerful opportunities for teachers to solve problems, develop 
curricula, and so on. 

What is important in planning and in funding, regardless of the technology selected, is to 
connect technology to powerful learning paradigms. Such designs ideally allow students and 
teachers to (1) work on authentic, meaningful, and challenging problems; (2) interact with 
the data in user friendly ways that allow some student control of learning; (3) build 
knowledge together within a learning community that is broader than a few students or 
schools with similar characteristics and interests; and (4) interact with practicing 
professionals and community members. 

Many schools can begin their involvement with technology-supported initiatives by 
investing in low-end technologies with high learning options. However, following Collins 
(1993), we strongly recommend that schools become involved as soon as possible in pilots 
with some of the types of research-based providers identified in this paper. It is important 
that these technology projects involve a significant group of teachers, and financial 
commitment as well; otherwise, it is not possible to experience what it means to use 
technology effectively for communication and for learning. Such projects should allow for 
experimentation with different models of instruction and different approaches to technology. 
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During this experimentation phase, schools can evaluate cost effectiveness in terms of the 
school's learning goals. If the school is not developing learning goals that link to Goals 
2000, the technology learning paradigm in this paper may serve as a resource. 

As the school's experience and expertise grow, it can progress to develop more powerful 
models of learning and reform using more complex technologies moving toward high- 
technology, high-learning options. To move in this direction, schools must, from the outset, 
build the capacity to move as a school toward technologies that are more connected and 
powerful It is also important for schools that may be in the high-technology and low- 
learning arena to consider new options. They might move from closed-system ILS and 
distance education technologies that are providing direct instruction toward more mteracUve 
technologies, open architecture, connectivity to distributed resources, and more engaged 
learning paradigms for existing ILS and distance education technologies. 

It is important to note that investing in technologies without investing in ongoing 
professional development, training, and support services is counterproductive and will 
ultimately be costly with limited payoff in learning. Successful technology programs with 
powerful impact are often supported by research-based agencies that focus on learning and 
collaboration. As business vendors seek to take on long-term roles, it will become important 
that they develop experience in research on learning and document evidence of student 
learning using their specific technologies and programs. 
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Summary 



This paper begins by examining the new consensus in educational research. It discusses the 
consensus against traditional models of learning. Then, it examines the emerging consensus 
on learning and the emerging consensus on technology performance, discussing and defining 
a variety of indicators of each. These are highlighted within the text. Next, these indicators 
are related to learning and technology interactions and school-based policy through the use of 
various tables and frameworks. Local, state, and national policy issues crucial to engaged ^ 
learning are highlighted next. From there, four changes in learning and educational reform 
are addressed. Problems that have arisen are mentioned and a proposed approach to solving 
these problems follows. The paper concludes by discussing possible next steps for research 
that allow the goals of engaged learning and high technology performance to be met. 

The purpose of this paper is to describe effective technology use to support engaged 
learning. Toward that end, we conceptualize learning as a continuum from passive to 
engaged and technology as a continuum from low to high performance; that is, a continuum 
from low-performance technology that focuses on low-level objectives and interactions and 
uses a transmission model of learning to high performance technology that supports authentic 
tasks and powerful student roles such as explorer, cognitive apprentice, teacher, and 
producer. When we look at the intersection of these two continua, we can describe 
technologies and technology-enhanced programs in terms of the extent to which they support 
engaged learning, or learning by doing challenging, authentic tasks. 

Overview of the Framework 

Part 1 of the paper provides an analytic framework. We first defined technology 
effectiveness in terms of a successful interface between learning and technology. The 
intersection of the two continua displaying passive to engaged learning and low- to high- 
performance technology created a 2 x 2 matrix or fourfold table. The four patterns 
represented in the table are Engaged Learning/ High Technology (Category A), Engaged 
Learning/Low Technology (Category B), Passive Learning/Low Technology (Category C), 
and Passive Learning/Low Technology (Category D). (See Table 1, The Learning and 
Technology Interface, Appendix A.) The table demonstrates that technology is effective 
when its software design, hardware configuration, and operation is designed for engaged 
learning and when the technology maximizes access and use of interactive multimedia 
resources networked to a global learning community. In other words, engaged learning is 
only part of the picture; high-performance technologies have various capabilities that can 
redefine and extend the parameters of engaged learning. 



ERLC 



59 



6, 



Operationally, engaged learning with high technology would include technologies and 
programs such as: 

• Networked projects with challenging tasks, access to the Internet, integrated multimedia 
capabilities including CD-ROM, two-way video conferencing, and access to professionals 

• Distance education networked with computers, challenging tasks, links to work with 
practicing professionals 

• Advanced, interactive multimedia tools and museum exhibits that promote thinking 

In contrast, technologies and programs that include both high technologies, such as 
integrated learning systems, and traditional distance education programs are associated with 
passive learning because they (1) are used to transmit information from a central source for 
the student to "learn," (2) are focused on low-level objectives, and (3) do not link students to 
global resources. Similarly, computer-based instruction, video and audio used to transmit 
information (e.g., talking head), and computer literacy courses teaching programming skills 
as an end in itself are usually focused on low-level objectives and involve central source 
technologies. 

This paper argues that a major goal of education should be to get as many schools as 
possible into Category A. Four different paths/trajectories allow us to reach this objective: 
Type I trajectory moves from Category D to B, Type H from B to A, Type m from C to A, 
and Type IV from D to A. This analytical framework can provide direction for 
policymakers wanting to use technology as a tool for classroom and school restructuring. 

The New Consensus 

When technology effectiveness is conceptualized as an intersection between learning and 
technology, it is possible to provide specific indicators of engaged learning and high- 
performance technologies that promote learning. Toward that end, Part 2 of this paper 
examines the research on learning and technology and finds increasing agreement. A striking 
feature of research is a strong reaction against traditional models of learning and technology 
and in favor of more engaged, authentic learning with collaboration. From this analysis, we 
developed 26 indicators of engaged learning and 25 indicators of high- performance 
technology. 

Consensus Against Traditional Models and Definitions. There is a growing concern 
that traditional models of learning are not adequate to meet the needs of the 21st century and 
have thus become outmoded. It is clear that we need citizens who can think strategically to 
solve problems, learn in a constantly changing environment, build knowledge from a wide 
range of sources, understand systems in diverse contexts, and collaborate both locally and 
globally using technology. Such attributes contrast greatly with the low-level basic skills and 
content often being taught using the transfer/transmission model of instruction. Traditional 
approaches to measuring technology and cost effectiveness by comparing a technology- 
enhanced program to a traditional model of learning are inadequate. Technology 
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effectiveness must be detennined in relation to the extent to which it supports and extends 
engaged learning and collaboration. Cost effectiveness must be judged laterally, so to speak, 
in terms of comparing the cost of the various technologies and programs that are proposed to 
meet the learning goals. Once a technology plan has been established that has reasonable 
costs, relative to the goal, the cost effectiveness must be judged in terms of getting the best 
purchase for equipment, software, training, and services. 

Emerging Consensus on Learning. Across various strands of research, there is 
increasing consensus on what constitutes the key variables of learning and instruction and on 
what defines engaged learning in the classroom and school. Jones (1992) has synthesized the 
research and identified eight variables of instruction, which were updated for the purposes of 
this paper: (1) the vision of learning, (2) the tasks that define the nature and level of 
achievement, (3) the assessment principles and practice, (4) the instructional model, (5) the 
characteristics of the learning context, (6) building classroom organization, (7) the learner 
roles, and (8) the teacher roles. 

To develop specific indicators within each of these categories of learning and instruction, 
we looked at basic research and theory as well as syntheses of that research base and 
find many commonalities; e.g., the American Psychological Association (1993), Brooks and 
Brooks (1993), and Means, et al. (1993). From this rich set of resources, we selected the 
indicators developed by Means and her colleagues as a guide to develop specific indicators of 
engaged learning within each of the eight categories in the analytic framework. Specifically, 
Means and her colleagues identified seven indicators to measure what they called effective 
learning and reform instruction: authentic and multidisciplinary tasks, performance-based 
assessment, interactive modes of instruction, heterogeneous groupings, collaborative work, 
student exploration, and teacher as facilitator. The descriptions of the indicators for each of 
the eight variables in our framework follow. (See also Table 2: Engaged Learning and 
Reform Instruction, Appendix A.) 

lo Indicators: Vision of Engaged Learning. What does engaged learning look like in 
students? Successful, engaged learners are responsible for their own learning. These 
students are self-regulated and able to define their own learning goals and evaluate their own 
achievement. They are also energized by their learning; their joy of learning leads to a 
lifelong passion for solving problems, understanding, and taking the next step in their 
thinking. These learners are strategic in that they know how to learn and are able to transfer 
knowledge to solve problems creatively. Engaged learning also involves being collaborative, 
that is, valuing and having the skills to work with others. 

2. Indicators: Tasks for Engaged Learning. In order to have engaged learning, tasks 
need to be challenging, authentic, and multidisciplinary. Such tasks are typically complex 
and involve sustained amounts of time. They are authentic in that they correspond to the 
tasks in the home and workplaces of today and tomorrow. These tasks often require 
integrated instruction that incorporates problem-based learning and curriculum by project. 
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3. Indicators: Assessment of Engaged Learning. These assessments involve presenting 
students with an authentic task, project, or investigation, and then observing, interviewing, 
and/or examining their artifacts and presentations to assess what they actually know and can 
do. This is called performance-based assessment. This assessment is generative in that it 
involves students in generating their own performance criteria and playing a key role in the 
overall design, evaluation, and reporting of their assessment. The best performance-based 
assessment has a seamless connection to curriculum and instruction so that it is ongoing. 
Assessment should represent all meaningful aspects of performance and should have equitable 
standards that apply to all students. 

4. Indicators: Instructional Models and Strategies for Engaged Learning. The most 
powerful models of instruction are interactive— instruction actively engages the learner— and 
generative— instruction encourages the learner to construct and produce knowledge in 
meaningful ways. Students teach others interactively and interact generatively with their 
teacher and peers. This allows for co-construction of knowledge, which promotes engaged 
learning that is problem-, project-, and goal-based. Some common strategies included in 
engaged learning models of instruction are individual and group summarizing, means of 
exploring multiple perspectives, techniques for building upon prior knowledge, 
brainstorming, Socratic dialogue, problem-solving process, and team teaching. 

5. Indicators: Learning Context of Engaged Learning. For engaged learning to happen, 
the classroom must be conceived of as a knowledge-building learning community. Such 
communities not only develop shared understandings collaboratively, but also create 
empathetic learning environments that value diversity and multiple perspectives. These 
communities search for strategies to build on the strengths of all of its members. Truly 
collaborative classrooms, schools, and communities encourage students to ask hard questions, 
define problems, lead conversations, set goals, have work-related conversations with adults in 
and out of school, and engage in entrepreneurial activities. 

6. Indicators: Grouping for Engaged Learning. Collaborative work that is learning- 
centered often involves small groups or teams of two or more students within a classroom or 
across classroom boundaries. Heterogerrous groups (inducing different sexes, cultures, 
abilities, ages, and socioeconomic backgrounds) offer a wealth of background knowledge and 
perspectives to different tasks. Flexible grouping, which allows teachers to reconfigure small 
groups according to the purposes of instruction and incorporates frequent heterogeneous 
groups, is one of the most equitable means of grouping and ensuring increased learning 
opportunities. 

7. Indicators: Teacher Roles for Engaged Learning. The role of the teacher in the 
classroom has shifted from the primary role of information giver to that of facilitator, guide, 
and learner. As a facilitator, the teacher provides the rich environments and learning 
experiences needed for collaborative study. The teacher is also required to act as a guide, a 
role that incorporates mediation, modeling, and coaching. Often the teacher also is a co- 
learner and/or co-investigator with the students. 
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8. Indicators: Student Roles for Engaged Learning. One important student role is that of 
explorer. Interaction with the physical world and with other people allows students to 
discover concepts and apply skills. Students are then encouraged to reflect upon their 
discoveries, which is essential for the student as a cognitive apprentice. Apprenticeship takes 
place when students observe and apply the thinking processes used by practitioners. Students 
also become teachers themselves by integrating what they've learned. Hence, they become 
producers of knowledge, capable of making significant contributions to the world's 
knowledge. 

Emerging Consensus on Technology Performance. Upon looking at technology 
through the lens of learning, six analytic categories were developed to examine technology 
performance: (1) school access to diverse technologies and resources both beyond the school 
and within a given classroom, (2) operability, (3) location and direction of resources, (4) 
capacity to engage students in challenging learning, (5) ease of use, and (6) functionality. 
TLrfe is, again, a strong consensus among researchers that technology artifacts can be 
designed to promote engaged learning. There are indicators within each of the six 
technology performances that would yield high performance and would promote engaged 
learning and reform instruction. (See also Table 3: High Performance Technology and 
Engaged Learning, Appendix A.) 

1. Indicators; Access. A vital indicator of high-performance technology is its access. A 
technology has high-performance in terms of access when it has connectivity, ubiquity, and 
interconnectivity. Connectivity refers to the capability of technology to access rich resources 
within and beyond the school because it is connected to those resources. Ubiquity means that 
technology equipment must be everywhere within the district so that all teachers and students 
can access and use it as a learning tool. Interactivity refers to the interaction that occurs 
when students and teachers actually communicate and collaborate in diverse ways. 
Additionally, it is imperative that there is a powerful design for equitable use of technology 
so that all students have access to the learning tools that provide high-quality, challenging 
opportunities to learn. 

2. Indicators: Operability. Once there is proper access, operability must be considered. 
A technology has maximal performance in terms of operability when it has interoperability, 
"the capacity to easily exchange data with and connect to other hardware and software to 
provide the greatest access tc all students. " To achieve this, it is necessary to have open 
architecture, which allows users to access data using different hardware and software, and 
sometimes lets them modify the system. Interoperability also requires transparency, or the 
ability to move easily from one format or program to another. This feature promotes 
engaged learning by allowing users to spend maximal time using the resources for multiple 
tasks rather than engaging in complex procedures for moving among formats/programs. 

3. Indicators: Organization of resources. These indicators refer to where 
information/data is stored, how resources are connected, how new resources get into the 
system, and who is in charge. In order to attain both high performance and engaged 
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learning, resource systems should be distributed rather than centralized. Distributed 
resources are socially constructed through collaborative efforts. They are designed for user 
contributions. That is, information can be contributed to the system from multiple sources so 
that many users can share the data. Moreover, such systems typically involve tools that are 
designed for collaborative projects, such as on-line conferences, bulletin boards, and access 
to remote files. These capabilities help in creating a knowledge-based community. 

4. Indicators: Design Features for Engaged Learning, Technology should include design 
features that promote engaged learning. One such feature is for the software itself to provide 
challenging learning tasks, opportunities, and experiences. This includes links to museums 
and libraries, access to expert and community members, use of rich media such as 3-D and 
virtual reality, and tools for interactive browsing. Another feature is software that allows 
students to learn by doing, using goal-based scenarios, problem-based learning, and 
simulations. Such tools allow the user to plan, reflect, make decisions, experience the 
consequences of actions, change directions, and examine alternative solutions and 
assumptions. In addition, the system should provide guided participation 9 , for example, the 
capacity to customize the content or to allow students to share their files with their peers. A 
final indicator is the capacity of the system to provide information that is just in time and just 
enough. 

5. Indicators: Ease of Use* It is essential that the programs be user-friendly and that they 
include effective help opportunities. It is also necessary that the system be speedy and 
provide feedback. Along these lines, user control of the system is important to access tools 
and use them however needed. Training and support to use the technology is vital and 
should be available from both local and remote locations. These ease-of-use features help the 
learner access the resources and the learning experiences offered by the system. 

6. Indicators: Functionality. Technology should be able to prepare learners for the 
diversity of technology functions that are basic to learning in the 21st century. Students 
should make frequent use of multimedia technologies and generic tools and have 
opportunities to use work-specific technologies. Technology should also prepare students to 
use tools that create tools, as well as provide opportunities to learn programming and 
authoring skills to create new programs and tools for others to use. Software should likewise 
allow students to develop skills related to project design and implementation such as setting 
goals, developing budgets, preparing analyses, and so forth. 

Learning and Technology Interactions and School-Based Policy. Technologies and 
technology-enhanced programs need to be measured by the extent to which they support 
engaged learning and educational reform. To this end, we have identified two ways of 
analyzing the intersection of learning and technology: develop tools to score the indicators 
and apply the concepts of engaged learning and high-performance technologies to examine 
the intersection in broad terms. To accomplish the first approach, we have developed a 
series of empowering tools— which are included in the body of the paper and in the 
Appendix— that schools can use for a variety of purposes: to define their current realities 
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and then use this assessment to help evaluate and design future goals of where they want to 
go from there; and to compare one program to another. In broader terms, technologies such 
as e-mail, computer-aided instruction, integrated learning systems, and distance learning can 
be examined from the perspective of engaged learning and high performance technology to 
evaluate their capacities, designs, and uses in school. These examinations of technology 
reiterate the point that technology effectiveness should be defined in terms of how well it 
supports engaged learning and instructional reform. 

Local, State, and National Policy Issues. In order for schools to use technology to 
promote engaged learning, certain elements inside and outside of the classroom must be in 
place. Five important policy issues have been identified that greatly affect a school's ability 
to employ technology in classrooms to promote engaged learning experiences. These include 
universal participation, high-quality technology access for all students, student familiarity 
with workplace technologies, commitment to ongoing professional development, and the role 
of the parents. 

The first policy issue concerns equity or universal participation: every student in every 
school must have access to and active involvement in the emerging information highway that 
will connect them with the world. It is crucial that this connection be implemented at a local 
level under the guidance of local policymakers. There are, however, notable barriers that 
prevent school participation, such as a lack of (1) focus and time for quality technology 
training, (2) models of instruction that promote engaged learning and effective use of 
technology, and (3) school architecture that supports a community of learners. It is important 
for policymakers to consider these barriers as they develop legislation. 

A second policy issue lies in providing high quality technology access for all students to 
achieve high standards of academic excellence. Student use of quality resources will not 
happen without an ongoing commitment to developing high standards for technology access 
and use through specific policies and financing strategies. Major barriers to implementing 
these policies at the local level are (1) local assessments that focus on low level and 
conventional objectives, (2) fragmented technology initiatives, and (3) tracking systems that 
separate students and technology into low- and high-level applications. 

The third set of policy issues focuses on the coordination of technology choices with uses 
in the workplace. The transition from school to work can be greatly strengthened by 
allowing students to become familiar with workplace technologies. Experience using 
technology with high transfer to the community and workplace is important. 

The fourth policy issue surrounds commitment to ongoing professional development that 
prepares educators to implement the proposed instructional strategies. This has important 
implications for building planning and classroom management as well as seamless 
curriculum, instruction, and assessment. 
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One final important issue is the role of technology for parents. Historically, groups of 
parents across the nation have not supported initiatives focused on teaching thinking. When 
this fear is coupled with technology, parents could feel very threatened if they are not 
brought into the partnerships. 

Learning, Educational Reform, and Technology 

Part 3 of the paper examines four changes that are defining educational reform. First, the 
strong movement to create a National Information Infrastructure and open up the Internet to 
schools has created significant resources for curriculum, instruction, and assessment models 
for schools and for professional development. Second, inherent in the growth of the Internet 
is the emergence of new content and service providers (such as government departments, 
public and private agencies, universities, ?<nd broadcasters). Through these providers, 
networked schools can receive up-to-the-minute data from around the globe and ongoing^ 
professional development support, as well af, import curriculum frameworks and materials to 
construct their own projects and curricula. 

Third, the growing skepticism about the use of standardized tests has led to a growing 
focus on student performance on authentic tasks and projects in the context of real audiences. 
One important new concept for technology and program evaluation is the notion of national 
testbeds, where teachers, students, scientists, educational researchers, and administrators 
work together to develop expertise and to evaluate the costs and benefits of a given 
technology or program. There is increasing evidence that technologies and technology 
programs that promotes engaged learning make a significant difference on curriculum, 
student achievement, professional development, and restructuring. Indeed, good, research- 
based technology programs seem to accelerate both classroom and school restructuring. 

Fourth, renewed interest and funding in school-to-work transition is linking workplace 
technologies with secondary and postsecondary educational experiences. Also, it is vital to 
redefine technology literacy in terms of preparation of students for the technology functions 
such as databases and spreadsheets needed in the workplace, community, and home. 

Regionally as a Unit for Learning/Technology Paradigms 

Part 4 of the paper examines the role of these changes in technology for education as a 
public institution. In particular, we examine the problems that technology creates and 
propose some solutions using the concept of regionality as the unit for change. 

Universal Access and Technology Growth as Problems for Education. There is no 
question that with the changes redefining educational reform, there is a need to develop an 
effective human and technology infrastructure that focuses on engaged learning. Education 
needs to be at the table with business, the telecommunications industry, the power 
companies, and others when vital issues of access to information are decided. Questions 
need to focus on how to configure technology to support engaged learning and collaboration 
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for all students— the equity issues; how all schools can have economical, two-way access to 
important public databases and resources; how can school regulations be changed to provide 
training and ongoing professional development, not just for teachers and administrators but 
also for librarians and technology specialists; and how can institutions that relate to schools 
change to provide better support so that schools use the available resources and participate as 
co-investigators and producers of knowledge and technology. 

Technologies are expected to reduce inequities in learning because in theory they can 
provide universal access to information. However, problems can arise in many poor schools, 
which do not have the funds, curricula, and/or faculty to participate in technological 
advancement. These schools are often hindered by their own bureaucracies that keep 
communication and development from moving beyond the walls of the schools into business 
and community sectors. It is possible, therefore, that richer schools that can access 
information and resources will become "information richer 11 while the poorer schools, in 
effect, become "information poorer." 

Four related problems also exist. First, there are limitations to the factory model of 
schooling in curriculum, instruction, and assessment. Top-down management and high 
volume have served as the economy of public education as an institution. Second, systematic 
reform in the cities has long been challenged by school funding formulae that depend on 
residential property taxes. If education is to change, the tax and funding structures of 
schooling must be a part of that change. Third, the scale up challenge will probably require 
the formation of new organizations and consortia, It seems unlikely that any one institution 
or agency will be able to provide very large-scale technology services to school and still 
maintain quality services. Fourth, there are broader economic reasons to think about 
strategies that go beyond a given city or state. The purchasing power of a multistate agency 
is far greater than that of a single state. 

A Proposed Approach for Scaling Up and Moving Out. With these problems in 
mind, a proposed solution is to form regional initiatives, which go beyond the school, 
district, or state level. Such infrastructures could facilitate opportunities for shared costs, 
broader learning opportunities, more equitable distribution of resources, and the strength that 
education needs to become a major player in today's growing technological society. These 
capabilities are beyond the scope of the Internet. What is needed is a set of regionally-based 
agencies to coordinate the evolution of technological trends. Such agencies would not only 
help restructure schools to coordinate technology and learning, but they would also plan and 
monitor the massive economic changes such a redistribution of services could yield. There 
are several possible strategies for developing innovative regional agencies: 

Regional Info-Port. One strategy is to create regional information distribution and 
coordination centers that would involve a diversity of players using the Internet and other 
free telecommunications. Such a port would then transport low-cost and/or free resources to 
schools, support school collaborations, link schools to scientists and community members 
around the world, work with schools to develop technology plans and higher education to 
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develop training and professional development programs, and provide ongoing support for 
school restructuring to promote engaged learning. 

Regional School Service Cooperative. In addition to an information port, there would 
be a need for service outreach— regional school service cooperatives. These would help 
schools access and use their resources, promote equitable access to technology-enhanced 
learning opportunities, develop regionally based and supported learning communities, develop 
technology evaluation designs, develop new policies, provide training and support services, 
and generate learning experiences for professional development. 

Regional Service Universities. A third possibility is a regional service university, 
which would provide services to schools and school networks subscribing to mutual programs 
and goals. Course* available from contributing members might emphasize using research 
and technology for systemic, long-term school restructuring. 

Educational Enterprise Communities. Increasingly, we are looking beyond individual 
schools to schools and communities as units of change. Several initiatives already exist as 
models such as the charter school movement and municipal efforts such as Baltimore Reads. 
Then, we must also look beyond cities and states to regions. The Learning Zone legislation 
being developed seems promising. 

Info-ports, service cooperatives, and regional universities are about restructuring access 
to quality information resources and collaboratively redefining education as a public 
institution. They are about building enterprise webs that serve to build new economic and 
political bases for education. 

Next Steps for Research and Education 

The next step for research and education will focus on some basic needs. There is a need 
for development of rich databases that describe successful technologies and programs as well 
as a communication system that will provide high-level research information for 
policymakers and a detailed database for researchers and educators. A second need relates to 
training and professional development. Right now schools are heavily dependent on 
commercial vendors for advise on what technologies to purchase and how to configure them 
as well as for training. There are few places they can access for research-based information 
about technology and engaged learning, the power and resources of the Internet, and useful 
advise about building architecture and requirements to support 21st century technology. 
Thus, there is a great need to disseminate information about these databases and tools that 
help schools address these issues and to evaluate technology and learning. 

Recommendations for Policymakers and Educators 

Some recommendations regarding technology policy for schools and other contexts emerge 
from this paper. First, strong support should be given to technology designs that empower 
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learning. It is also important to support movement toward distributed networks as opposed 
to central source providers. In terms of planning and funding, it is important, regardless of 
the technology selected, to connect technology to powerful learning paradigms that allow 
students and teachers to work on authentic and challenging problems, interact with the data 
in ways that allow student control, build knowledge within a learning community, and 
interact with practicing professionals and community members. 

It is recommended that schools become involved as soon as possible in pilot programs 
with research-based providers. Also, in order to move toward high-technology, high- 
learning options, schools must build the capacity to move toward technologies that are more 
connected and powerful. Finally, it is important to note that investing in technologies 
without investing in ongoing professional development, training, and support services is 
counterproductive and will ultimately be costly, with limited payoff in learning. These 
recommendations will allow our schools to use technology to its fullest potential and, hence, 
move toward a more successful learning endeavor. 
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APPENDIX A 

TABLES DEFINING THE INTERSECTION OF LEARNING AND TECHNOLOGY 



For the convenience of the reader, Tables 1-3 have been reprinted here with glossaries. All 
three tables related to the interaction of learning, defined as a continuum from passive to 
engaged, and technology defined as a continuum from high performance technology that 
promotes engaged learning to low performance technologies that do not. Table 1 shows the 
four categories that emerge when these two variables intersect. Table 2 provides indicators 
for eight categories of engaged learning with a Glossary defining the 26 indicators. Table 3 
provides indicators for high performance technologies that promote learning with a Glossary 
defining the 25 indicators. 



The form below is designed to assist the user to focus on the appropriate unit of analysis: 

Name of User: 

Unit of Analysis: (check one) 
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TABLE 1 



THE LEARNING AND TECHNOLOGY INTERFACE 



ft 

s 



3 



(C) 



Examples 



Closed integrated learning systems 
focusing on low level objectives and 
standardized, objective assessments 

-Traditional distance education used to 
transmit information from a central source 
and focused on low level objectives and 
assessments (talking head) 

-Connections to homes that are linked only 
to closed networks for the school and 
vendor and perhaps to other schools using 
the same vendor 



(D) 



Examples 



-Computer-based instruction/drill and 
practice focusing on low level objectives 

-Instructional television focused on low 
level objectives 

-Video and audio used to transmit 
information as a lecture or talking head 

-Teaching a computer language or word 
processing as an end in itself as technology 
literacy 



(A) 



Networked projects with: challenging 
tasks, access to Internet, integrated 
multimedia capabilities including CD- 
ROM, two-way video conferencing, access 
to professionals 

- Distance education networked with 
computers; challenging tasks; linked to 
work with real-world professionals and 
data; two-way video 

-Advanced tools and high-technology 
museum exhibits that are interactive and 
support high-level thinking 



(B) 



Examples 



-Projects using multimedia experiences and 
data provided by CD-ROM for authentic 
and challenging learning 

-Local file sharing allowing students access 
to all files for communal editing and 
development 

-E-mail for inquiry collaborations 

-State network support for schools using 
the Internet for projects 



Passive Learning 



Engaged Learning 



LEARNING 
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TABLE 2 



ENGAGED LEARNING AND REFORM INSTRUCTION 



VarlaDlca 01 lueSming 

and Instruction 


jjiiiicuiur& 1 or ruugitgcu jlaj<4i iiiiig 
and Reform Instruction 


Vision of Learning 


Responsible for learning 

duaieglC 

Energized by learning 
Collaborative 


Taslcs 


Authentic* 
^flaiicnging 

Integrative/Interdisciplinary* 




x CllUI illdJlCC "UdSCU 

Generative 

Seamless and ongoing 
Equitable 


Instructional Modes 


Interactive* 
Generative 


learning v^uniCA.L 


v^UXXaUUIclLlVC 

Knowledge building 
Empathetic 


\jluuplll5 


Equitable 
Flexible 


Teacher Roles 


Facilitator* 
Guide 

Co-Learner/Co-Investigator 


Student Roles 


Explorer* 

Cognitive apprentice 

Teacher 

Producer 



Copyright ® 1994 The North Central Regional Educational Laboratory 
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GLOSSARY FOR TABLE 2 



VISION OF LEARNING 

Responsible for Learning - learner involved in setting goals, choosing tasks, developing assessments and standards for the tasks; 
has big picture of learning in mind 

Strategic - learner actively develops repertoire of thinking/learning strategies 

Energized by learning - learner is not dependent on rewards from others; has a passion for learning 

Collaborative - learner develops new ideas and understanding in conversations and work with otficrs 

TASKS 

Authentic - pertains to real world, may be addressed to personal interest 

Challenging - difficult enough to be interesting but not totally frustrating, usually sustained; requires creative and or critical 
thinking 

I ntegrative/Inter disciplinary - involves integrating disciplines to solve problems; address issues 
ASSESSMENT 

Performance-based - involving a performance or demonstration f usually for a real audience and useful purpose 
Generative - learner constructs the knowledge and artifacts assessed, ideally learner generates performance criteria and 
contributes to overall assessment plan 

Seamless and ongoing - assessment is part of instruction and vice versa; students learn during assessment 
Equitable standards - assessment is culture fair and standards apply to all 

INSTRUCTION/MODEL 

Interactive - teacher and students actively engaged in learning with each other and with instructional resources 
Generative - instruction oriented to constructing meaning; providing meaningful activities/experiences 

LEARNING CONTEXT 

Collaborative - instruction conceptualizes students as part of learning community; activities are collaborative within and across 
classroom boundaries 

Knowledge building - learning experiences set up to bring multiple perspectives to solve problems such that each perspective 

contributes to shared understanding for all; goes beyond brainstorming to construct meaning 

Empathetic - learning environment and experiences set up valuing diversity, multiple perspectives, strengths 

GROUPING 

Heterogeneous - small groups involve persons from different ethnic cultures and backgrounds, genders, and abilities 
Equitable - small groups organized so that over time all students have challenging learning tasks/experiences 
Flexible - different groups organized for different instructional purposes so each person is member of different groups based on 
need and/or interests 

TEACHER ROLES 

Facilitator - stimulates and monitors discussion and project work but does not control; negotiates with students and others 
Guide - helps students to construct their own meaning by modelling, mediating, explaining when needed, redirecting focus, 
providing options 

Co-Learner/Co-Investigator - teacher considers self as learner; willing to take risks to explore areas outside his or her expertise; 
collaborates with other teachers, students, and practicing professionals 

STUDENT ROLES 

Explorer* - students have opportunities to explore new ideas, new tools; push the envelope in ideas and research; engages in 
frequent discovery-oriented, open-minded activities 

Cognitive Apprentice - learning is usually situated in relationship with mentor who coaches students to develop ideas and skills 
that simulate the role of practicing professionals (e.g., engage in real research); student observes, applies, and refines through 
practicing the thinking processes of the practitioner 

Teacher - students encouraged to teach others in formal and informal contexts 
Producer - students develop products of real use to themselves or others 
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TABLE 3 



HIGH PERFORMANCE TECHNOLOGY AND ENGAGED LEARNING 



Variables of Technology 
Performance 


Indicators for High Technology Performance -> 
Engaged Learning 


Access 


Connectivity 

Ubiquity 

Interactivity 

Design for equitable use 


Operability 


Interoperability 
Open architecture 
Transparency 


Resource Organization 


Distributed resources and logic/intelligence 

User contributions 

Design for collaborative projects 


Engagement 


Opportunities for challenging tasks and experiences 
Opportunities to learn by doing 
Guided participation and intelligent tutoring 
Information just in time and just enough 


Ease of Use 


Effective helps 
User friendly 
User control 
Training and support 
Speed 


Functionality 


Use of multimedia technologies 
Use of generic tools 

Use of context-specific (work-specific) tools 

Programming/authoring skills 

Design and project implementation skills 

Use of tools that are used to make tools/programs 
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GLOSSARY FOR TABLE 3 



ACCESS 

Connectivity — schools connected to Internet and other resources 

Ubiquity - diverse technologies (computers, printers, facsimiles, video) near where students and teachers need them; 
available to critical mass per classroom 
Interactivity - refers to active use of technology 

Design for equitable use - planning for all students to access and use technology and programs that are high quality 
and challenging 

OPERABILITY 

Interoperability - capability to exchange data amon^ diverse technologies easily 

Open architecture - allows users to access "third party " software; contrasts with closed systems 

Transparency — users not aware of procedures to change programs and data 

RESOURCE ORGANIZATION 

Distributed resources - assumes that intelligence and resources reside in diverse locations; contrasts with centralize 
system that controls all resources 

User contributions — users can provide input/resources to the system on demand 

Design for collaborative projects - designed to facilitate communication among users with diverse technologies; not 
communication with central source 

ENGAGEMENT 

Opportunities for challenging tasks and experiences - software provides tasks/data and learning opportunities that 
stimulate thought and inquiry 

Opportunities to learn by doing - software provides simulations, goals based learning, real-world problems 
Guided participation and intelligent tutoring ~ system responds to student use intelligently; is able to diagnose and 
prescribe new learning 

Information just in time and just enough - random access; multiple points of entry; different levels and types of 
information 

EASE OF USE 

Effective helps - system provides help indices that are more than glossaries; may provide procedures for tasks and 
routines 

User friendly - user enjoys use; is not burdened by long delays and overly complex procedures 

Training and support- system provides both technology training and training in applications and ongoing support 

Speed — system not down for long delays 

FUNCTIONALITY 

Use of multimedia technologies - e.g., color printers, integrated media driven by computer, video, audio, fax 

Use of generic tools - e.g., spreadsheets, databases, word processing 

Use of context specific tools - work specific technologies; e.g., computer-assisted drawing 

Programming and authoring skills ~ teaches students how to create their own tools and programs 

Technology-related design and project implementation skills - software that helps users comprehend and use 

skills for designing and implementing different kinds of projects (e.g., project scheduling and budgeting) 

Use of tools that are used to make tools/programs ~ system provides wizards and other tools that are used to make 

tools 
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Version 1.0 



APPENDIX B: COMPARING CURRENT REALITIES AND FUTURE GOALS 

Appendix B is a series of three tables to use in various contexts to plot current realities and 
future goals with regard to actual practice (what is in place in the classroom and school with 
regard to technology use) and with regard to policy (what is in place in plans, mission 
statements, curricula frameworks, assessment programs, and the like that determine school and/or 
district policy). The scoring system we have developed allows users to indicate the level of 
involvement or development with regard to that indicator at the practice level and at the policy- 
level. To elaborate what is meant by practice and policy, consider the following example 
comparing two hypothetical schools in terms of authentic tasks. 

Example: School A has only just begun developing such tasks through a very powerful 
technology program they are piloting (teaming practice). They have implemented the 
program as part of their new mission to become aligned with the learning and technology 
needs of the 21st century (learning and technology policy). In contrast, many teachers at 
school B have been developing authentic tasks through the school's assessment program 
(learning policy), and this development is a major strength. School B has almost no 
technology in place (technology practice), but it has become a major agenda in their school 
improvement plan (technology policy). 

Each table is shown twice: (1) a completed version using data from a hypothetical school and 
(2) a blank version so that users can have it for local applications. Tables 4, 5, and 6 have been 
completed using hypothetical data; Tables 4a, 5a, and 6a are blank. Each table is followed by a 
legend that tells users how to fill it in. It is very important to understand that Tables 4, 5, 
and 6 represent indicators only for Category A on Table 1: engaged learning that is braided 
with high performance technology. There is a point of view presented in this paper that says 
that every school or user should be working toward the ideal of very engaged learning that is 
supported and extended by high performance technology. 

It should be noted that these are tools for every school and every program , regardless of 
current status of that school or program with regard to learning and technology. Every school or 
program will have a mixed profile— some scores that indicate the presence of engaged learning 
and/or high performance technology practice and policies; and some scores that reflect lack of 
development. Moreover, we believe that most schools are in the midst of changing. 

Table 4 on the next page allows users to score where they are now in terms of what indicators 
would describe their current realities for learning and for technology. The data is hypothetical 
but reflects characteristics of schools as they exist today. The Legend for Table 4 shows the 
values for Practice and Policy Scores. Following that is an abbreviated scenario of how a 
hypothetical school would score itself on engaged learning and high performance technology. 
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TABLE 4: CURRENT REALITIES FOR SCHOOLS 
ENGAGED LEARNING AND HIGH PERFORMANCE TECHNOLOGY 





Practice 


Policy 




High Performance Technology 


Practice 


Policy 


Indicators 


Scores 


Scores 




Indicators * 


Scores 


Scores 


Vision of Learning 








Access 






Responsible for learning 


0** 


2 




Connectivity 


3 


2 




3 


3 




Ubiquity 


3 


2 


TTwroiT^H Kv lamina 


2 


1 




Interactivity 


3 


2 


Collaborative 


1 


1 




£>cjign for equitable use 


1 


2 


Tasks 








Operabfltity 






AUUlCUUv 


1 


3 




Interoperability 


3 


3 


CHall e ng ing * 


1 


1 




Open architecture 


3 


3 


Tnt wr*t i v^/Tnt«» ra i *r i nl i r\A rv * 

¥W liliVlUiaVipillMll J 


3 


3 




Transparency 


3 


3 


Assessment 








Resource Organization 






Pe rfo rm&nc e-baaed* 


2 


3 




Distributed resources 


3 


3 


Generative 


2 


1 




User contributions 


3 


3 


Seamless and ongoing 


1 


1 




Collsb. Projects/Design 


3 


3 


Ecjuitible standards 


2 


3 










Iustroction/Model 








Engagement 






Interactive* 


3 


3 




Opportunitiea for challenging 


3 


3 


Generative 


2 


2 




taaka and experiencea 


- 


- 










Opportunitiea to learn by doing 


2 


3 










Guided participation and 


2 


1 










intelligent tutoring 














Info, just in time and just enough 


1 


0 


Learning Context 








Ease of Use 






Collaborative* 


2 


3 




Effective helps 


3 


3 


Knowledge building 


0 


2 




User friendliness 


3 


3 


Empathetic 


2 


3 




User control 


3 


3 










Training and support 


1 


3 










Speed 


3 


3 


Grouping 








Functionality 






Hetcrogeneoua* 




3 




Use of multimedia technologies 


3 


3 


Equitable 


1 


3 




Use of generic tools 


2 


1 


Flexible 


2 


1 




Use of context-specific teo!a 


0 


0 










Programming and authoring alalia 


I 


2 










Deaign and project akilla 


2 


3 










Use of tools to make tools 


0 


0 


Teacher Rotes 














Facilitator* 


2 


3 










Guide 


1 


2 










Co- Learner/Co-investigator 


1 


1 










Student Roles 














Explorer* 


1 


3 










Cognitive apprentice 


3 


3 










Teacher 


0 


1 










Producer 


1 


0 










Total Scores 


42 


55 






57 


57 



Copyright (c) 1994 The North Central Regional Educational Laboratory 
♦Indicators provided by Means, et al., 1993 
**A11 numbers represent hypothetical scores. 
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Legend for Table 4: Current Realities 

Scores for Learning Practice axe marked on a scale from 0-3 

0 = Not in place at this time. 

1 = Some users/teachers exploring/piloting/developing. 

2 = Many users/teachers have good skills in these areas; practice is effective. 

3 = Most users/teachers have mastery, and practice is very widespread; it is a major strength for the school. 



Scores for Technology Practice are marked on a scale from 0-3 

0 = Not in place at this time. 

1 = Some users/teachers have equipment and are exploring/piloting/developing. 

2 = Many users/teachers have good computer and technology skills and are actively engaged with the technology. 

3 = Most users/teachers have mastered complex technologies (hardware and software) and effective use of technology to 
promote engaged learning and instruction is a major strength in the school or district. 

Scores for Policy are marked on a scale from 0-3. 

Policy Scores refer to what the school/community values for a given indicator. This value must he shown in some kind of 
policy such as a mission statement, a curriculum framework, an assessment system, huilding organization policy, or a 
design or plan that has been accepted. 

0 = not in place 

1 «= not so important 

2 = somewhat important 

3 = very important 

Rationale for Some Sample Practice and Policy Scores. This hypothetical school has a lot of thinking skills 
programs that are quite motivatiri, so they think they have some important things in place with regard to their Vfc'on of 
learning; that is, thinking of learners as Strategic and Energized hy learning. Accordingly, the staff gives Practice Scores 
of 3 and 2, respectively, to those Learning Indicators, and 3 and 1 for the Policy Scores. However, they noted in doing 
this self-assessment that they really did not have tfjnything in place that specifically addressed student Responsibility for 
learning in the ways defined here, even though student responsibility for learning is stated in two of the curriculum 
frameworks. So they gave a Practice score of 0 and a Policy Score of 2. Regarding Collaborative, there is much staff 
enthusiasm for what we at the school call "cooperative learning," which is highly structured, not the characteristics of 
collaborative learning given in the text of this paper. So staff gave themselves a Practice Score of 1 on this Learning 
Indicator because some teachers were experimenting with less structured cooperative learning paradigms. They gave 
themselves a 1 for the Policy Score because the Beliefs Statement the school developed last year referred to the need for 
"learning in group contexts." 

Moving to the next category, many of the staff would have liked to give themselves a 3 for all three Indicators of 
engaged learning for Ac Tasks variable. But the current reality is that while the staff talked about the need for tasks that 
were Authentic and Challenging, in fact only a few of the staff were developing such tasks, and they were not tasks that 
were as "rich" as those we described, so they decided that they should rate themselves Practice Scores of 1 for both 
indicators. The Integrative/Interdisciplinary was easy. They have had such projects for years so they gave themselves a 
Practice Score of 3. However, when it came to the Policy Scores, their curriculum frameworks talk a lot about Authentic 
tasks, so they gave themselves a 3, as was the case for Integrative/Interdisciplinary. But they really had no shared 
language for Challenging, so they gave themselves a Policy Score of 1. 

The paragraphs above should give the user a sense of how this table should be used-reflectively by a group of 
persons, thinking about what is in place ar,d what is valued in the shared artifacts of the school and comparing the 
definitions in the text here with policies and actual practice at the school. 
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TABLE 4a: CURRENT REALITIES 
FOR ENGAGED LEARNING AND HIGH PERFORMANCE TECHNOLOGY 



Engaged Teaming Indicator! 


Practice 
Scores 


Policy 
Scores 


Vision of Learning 
Responsible for learning 
Strategic 

Energized by learning 
Collaborative 






Tasks 

Authentic * 
Challenging* 

Integrative/Interdisciplinary * 






Assessment 

Performance-bated* 
Generative 

Seamless and ongoing 
Equitable standsrdt 






Instruction/Model 

Interactive* 

Generative 






Learning Contort 
Collaborative* 
Knowledge building 
Empathetic 






Grouping 
Heterogeneous* 
Equitable 
Flexible 






Teacher Roles 

Ftcilitator* 

Guide 

Co-Leamer/Co-investigstor 






Student Roles 
Explorer* 

Cognitive apprentice 

Teacher 

Producer 






Total Scores 







High Performance 

Technology 

Indicators 


Practice 
Scores 


Policy 
Scores 


Access 

Connectivity 

Ubiquity 

Interactivity 

Design for equitable use 






OperabOity 
Interoperability 
Open architecture 
Transparency 






Resource Orgsjuxarion 

Distributed resources 
User contributions 
Collab. Projects/Design 






Engagement 

Opportunities for challenging 

tasks and experiences 
Opportunities to learn by doing 
Guided psrticipstion and 

intelligent tutoring 
Info just in time and just 
enough 






Ease of Use 
Effective helps 
User friendliness 
User control 
Training and support 
Speed 






Functionality 

Use of multimedia technologies 
Use of generic tools 
Use of context specific toois 
Programming and authoring 
skills 

Design <& project skills 
Use of tools to make tools 






















I 



Copyright * 1994 The North Central Regional Educational Laboratory 
* Indicators provided by Means, et al., 1993 
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Legend for Table 4a: Current Realities 



Scores for Learning Practice are marked on a scale from 0-3 

0 = Not in place at this time. 

1 = Some users/teachers exploring/piloting/developing. 

2 = Many users/teachers have good skills in these areas; practice is effective. 

3 = Most users/teachers have mastery, and practice is very widespread; it is a major strength for the school. 

Scores for Technology Practice are marked on a scale from 0-3 

0 = Not in place at this time. 

1 = Some users/teachers have equipment and are exploring/piloting/developing. 

2 = Many users/teachers have good computer and technology skills and are actively engaged with the 
technology. 

3 = Most users/teachers have mastered complex technologies (hardware and software) and effective use of 
technology to promote engaged learning and instruction is a major strength in the school or district. 



Policy Scores refer to what the school/community values for a given indicator. This value must be shown in 
some land of policy such as a mission statement, a curriculum framework, an assessment system, building 
organization policy, or a design or plan that has been accepted. 

0 = not in place 

1 — not so important 

2 = somewhat important 

3 *= very important 
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TABLES: FUTURE GOALS 
FOR ENGAGED LEARNING AND HIGH PERFORMANCE TECHNOLOGY 



Engaged Learning 
Indicators 


Practice 
Scores 


Policy 
Scores 


Vision of Learning 
Responsible for learning 
Strategic 

Energized by learning 
Collaborative 


3** 

2 


1 


Tasks 

Authentic* 
Challenging* 

wtegrative/Interdisciplinary* 


2 

2 


2 


Assessment 

Performance-based* 
Generative 

Seamless and ongoing 
Equitable atandards 


1 
2 


1 
1 


Instruction/Model 

Interactive* 

Generative 


1 

1 




Learning Context 
Collaborative* 
Knowledge building 
Empathetic 


1 
1 
1 




Grouping 
Heterogeneou** 
Equitable 
Flexible 


1 

1 




Teacher Roles 

Facilitator* 

Guide 

Co-Leamer/Co-investigator 


1 




Student Roles 
Fxplorer* 

Cognitive apprentice 

Teacher 

Producer 


1 
1 




Total Scores 


22 


5 



High Performance Technology 
Indicators 


Practice I 
Scores 


Policy 
Scores 


Access 

Connectivity 

Ubiquity 

Interactivity 

Design for equitable use 






OperabiHty 

Interoperability 
Open architecture 
Transparency 






Resource Location & Dir. 
Distributed resources 
User contributions 
Collab. Projects/Design 






Engagement 

Opportunities for challenging 

tasks and experiences 
Opportunities to learn by doing 
Guided participation and 
intelligent tutoring 
Info just in time and just enough 






Ease of Use 
Effective helps 
User friendliness 
User control 
Training and support 
Speed 






Functionality 

Use of multimedia technologies 

Use of generic tools 

Use of context specific tools 

Programming and authoring skills 

Design and project skills 

Use of tools to make tools 


1 
2 
1 
2 
1 


















7 


0 



Copyright e 1994 The North Central Regional Educational Laboratory 
* Indicators provided by Mean*, et al., 1993 
** All data are hypothetical. 
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Legend for Table 5; Future Goals 



Explanation for Scoring: Some Hypothetical Examples for Learning Indicators 

Consider the indicators for Vision of Learning in Table 4 for Current Realities for our hypothetical school. 
Strategic was scored at 3 in Policy and Practice Scores, so this indicator probably should not be a priority for 
future goals. However, Energized by learning was scored at 2 for Practice and 1 for Policy, so it is a 
possibility for future growth. Similarly, Responsibility and Collaborative were given relatively low scores in 
Practice and to some extent in Policy. 

Which Vision of Learning indicator should get priority in Table 5. Future Goals? Referring back to Table 4 . 
staff are most concerned about the discrepancy between the Policy Score of 2 for Responsible for learning and 
the Practice Score of 0, an important imbalance. Further, they feel if they develop Collaborative practices, it 
will generate much Energy for learning. Accordingly, for Table 5. Future Goals , the school is given a Practice 
Score of 3 for Responsible for learning, and a Practice Score of 2 for Collaborative. Since Policy Scores are 
already high for Responsible for learning and Strategic, they decide to put a small emphasis on incorporating 
collaborative into a school Policy, so they score it 1. 

For Tasks in Table 4 for Current Realities, staff noticed another discrepancy: Authentic tasks are valued at 3 
in terms of Policy, but Practice was scored at 1. Moreover, there was a lot of emphasis on the need for tasks 
to be Challenging in this document. Clearly, the school's Practice and Policy Scores reflect their capacity for 
Integrative /Interdisciplinary, so there is no need to focus on that for the future. 

After much thought about the importance of Tasks that are Authentic and Challenging, they decide to put a lot 
of energy into building this capacity. So for Table 5 . under the Tasks category, they give a Practice Score of 2 
for Authentic and Challenging practices and a Policy Score of 2 for Challenging tasks. They think these 
policies and practices will drive many improvements in Assessment, Instructional Model, Teacher Roles and 
Student Roles , so they, feel that they want to concentrate their Future Goals on these indicators. 



Explanation for Scoring: Some Hypothetical Examples for Technology Indicators 

This hypothetical school has a lot of Policies in place as plans, mission statements, and belief statements. They 
also have a lot in place for all the Technology Indicators. So they decide that they will invest only in 
developing the Functionality category. Thus, they add 1 for the Use of generic tools; 2 to the Use of context- 
specific Tools; 1 for Programming;. 2 for Design and project skills; and 1 for Use of tools to make tools: a total 
of 7 points in the Practice column. They would like to have a parallel in Policy for this Indicator, but the 
school is awaiting important legislation on School-to-Work Policies, so they want to wait until they know more 
about that legislation before they develop new policies for the school. Thus, they will not be doing any 
development in Technology Policy at this time. 
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TABLE 5a: FUTURE GOALS 
FOR ENGAGED LEARNING AND HIGH PERFORMANCE TECHNOLOGY 



Engaged Learning 
Indicators 


Practice 
Scores 


Policy 
Scores 




High Performance Technology 
Indicators 


Practice 
Scores 


Policy 
Scores 


Vision of Learning 
Responsible for learning 
Strategic 

Energized by learning 
Collaborative 








Access 

Connectivity 

Ubiquity 

Interactivity 

Design for equitable use 






Tasks 

Authentic* 
Challenging* 

Integrative/Interdisciplinary* 








Operabflhy 

Interoperability 
Open architecture 
Transparency 






Assessment 

Performance-based* 

Generative 

Seamless and ongoing 
Equitable standards 








Resource Organization 
Distributed resources 
User contributions 
Collab. Projects/Design 






Instruction/Model 

Interactive* 

Generative 








Engagement 

Opportunities for challenging 

tasks and experience! 
Opportunitiea to learn by doing 
Guided participation and 
intelligent tutoring 
Info just in time snd just enough 






Learning Context 
1 Collaborative* 
Knowledge building 
Empathetic 








Ease of Use 
Effective helps 
User friendliness 
User control 
Training and support 
Speed 






Grouping 
Heterogeneous* 
Equitable 
Flexible 








Functionality 

Use of multimedia technologies 
Use of generic tools 
Use of context specific tools 
Programming and authoring skills 
Develops design & project skills 
Use of tools to make toola 






Teacher Roles 

Facilitator* 

Guide 

Co-Learner/Co-invettigator 














Student Roles 
Explorer* 

Cognitive apprentice 

Teacher 

Producer 














Total Scores 















Copyright • 1994 The North Central Regional Educational Laboratory 
* Indicators provided by Means, et si., 1993 
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Legend for Table 5a: Future Goals 



Some general principles for scoring: 

-If an Indicator is given a Practice Score of 3 (very important), it is already a major strength, so there is no 
need for growth for any Indicator scored at 3. Similarly, if an Indicator is given a Policy Score of 3, it should 
not be a priority for future growth. 

-If there is a large discrepancy between Policy and Practice Scores, the school should look at some efforts 
toward alignment. If the Policy Score of any given Indicator is 3, and the Practice Score is 0, it would be 
important to flag that Practice Score by scoring it a 1, 2, or 3 depending oh the beliefs and values of the school. 
Similarly, if a Practice Score is high and a Policy Score is low, the school should consider developing Policies 
to be in alignment with the Practice in order to maintain training support and administrative support for that 
Practice. 

-If the school's Policy and Practice Scores are higher for Learning, compared to Technology, or vice versa, 
these discrepancies should flag attention to the need for development in the area that scored lowest. 

The scores in Table 5 are calculated as follows, using the data from Table 4. 

1) The big picture for our hypothetical school is determined by looking at Table 4, Current Realities for Policy 
and Practice. Thus, the user reflects on individual scores for each indicator in Table 4, asking the question: do 
we want to grow in this area? 

2) The user decides if there should be more emphasis or priority on a given indicator for Future Goals, 
-If not, no additional score or value needs to be added . 

-If the school wants to grow in a given area, however, this is shown bv adding to Policy and Practice Scores in 
Table 5 . 

Clearly, this school is very strong on Policy and less strong on Practice. This is revealed by the large number of 
3s in the Policy Scores column of Table 5, compared to the number of 2s and Is in the Practice Scores column. 
So this hypothetical school decides it will focus its energies on making Practice more in alignment with Policy. 
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TABLE 6 

CURRENT REALITIES AND FUTURE GOALS 
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legend for Hy pothetical Data in Table 6 



Calculations for Current Realities Profile from Table 4 fo r Table 6 



Learning Practice Total Score 42 

Learning Policy Total Score + 55 

Learning/Current Realities Grand Total 97 

Place a circle on the horizontal axis for at about 97. 

Technology Practice Total Score 57 

Technology Policy Total Score + 57 

Technology/Current Realities Grand Total 1 14 

Place an X along the vertical axis at about 114. 



To plot the Current Realities profile: Draw a solid line upward from the circle on the horizontal axis (Learning) 
to intersect with a line to the right from the X on the vertical axis (Technology). 



Calculations for Future Goals Profile from Table 5 for Table 6 
Learning Practice Total Score 22 
Learning Policy Total Score + 5 

learning/Future Goals Grand Total 27 

Add the Learning Future Goals total from Table 5 calculations above to the Learning Current Realities total 
from Table 4: 97 + 27 = 124. Place a triangle on the horizontal axis at 124. 



Technology Practice Total Score 7 
Technology Policy Total Score + 0 



Technology/Future Goals Grand Total 7 

Add the Technology Future Goals total from Table 5 calculations above to the Technology Current Realities 
total from Table 4. 114 + 7 = 121. Place a square on the vertical axis at 121. 

To plot the Future Goals profile, draw a dashed line upward from the triangle on the horizontal axis (Learning) 
and to the right from square on the vertical axis (Technology) until thuy meet. 
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TABLE 6a 

CURRENT REALITIES AND FUTURE GOALS 
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Legend for Table 6a 



Calculations for Current Realities from Table 4a for Table 6a 

Learning Practice Total Score 

Learning Policy Total Score + 

Learning/Current Realities Total 

Place a circle on toe horizontal axis for the Learning/Current Realities Grand Total Score. 

Technology Practice Total Score 

Technology Policy Total Score + 



Technology/Current Realities Total 

Place an X along the vertical axis for the Technology /Current Realities Total Score. 

To plot the Current Realities profile: Draw a solid line upward from the circle on the horizontal axis (Learning) 
to intersect with a line moving to the right from the X on the vertical axis (Technology). 

Calculations for Future Goals from Table Sa for Table 6a 

Learning Practice Total Score 

Learning Policy Total Score + 



Learning/Future Goals Grand Total 

Add the Learning Future Goals total from Table 5 calculations above to the Learning Current Realities total 
from Table 4. 

+ ~ . 

Place a triangle on the horizontal a*ts where the total is. 



Technology Practice Score 

Technology Policy Score + 



Technology/Future Goals Grand Total 

Add the Technology Future Goals total from Table 5 calculations above to the Technology Current Realities 

total from Table 4: + = . 

Place a square on the vertical axis where the total is. 



To plot the Future Goals profile: Draw a d ashed line upward from the triangle on the horizontal axis (Learning) 
to intersect with a line moving to the right from the square on the vertical axis (Technology). 
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APPENDIX C: COMPARING TECHNOLOGY PROGRAMS 



The Tables in this Appendix are to compare two technologies or technology-enhanced 
programs. Users could be school staff seeking to compare how two programs would rate in 
terms of promoting the learning and technology features listed in Table 7. The scores for 
each indicator would be drawn from the descriptive or promotional literature. First the user 
completes Table 7 and then uses these scores to plot profiles in Table 8. 
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TABLE 7: COMPARING TECHNOLOGY PROGRAMS 

FOR ENGAGED LEARNING AND HIGH PERFORMANCE TECHNOLOGY 



Engaged Learning 
Indicators 


Program 

A Scores 


Program 
B Scores 




Dsgn Prtc 


i 

Dsgn Prtc 


Vision of Learning 

IcVsnnniihle fnr learning 

Strategic 

Energized by learning 
Collaborative 


1** 1 

0 0 
0 0 
2 1 


3 2 

2 1 

3 3 
3 3 


Tasks 

AtitHi*T\f ir*^ 

AUlUCUUb 

Challenging 4 

Integrative/Interdisciplinary* 


1 0 

2 0 
2 2 


3 3 
3 3 
3 3 


Assessment 

Performance-based* 

Generative 

S camleu and ongoing 


0 0 

0 0 

1 0 
0 0 


2 1 

3 3 
3 1 
2 1 


Instruction/Model 

Interactive* 

Generative 


2 0 
0 1 


3 2 
2 1 


Learning Context 
Collaborative* 
Knowledge building 
Empathetic 


2 2 
0 0 
0 0 


3 3 

2 1 

3 1 


Grouping 

Heterogeneous* 

Equitable 

Flexible 


3 1 
3 0 
3 1 


1 1 
1 1 
3 3 


Teacher Roles 

Facilitator* 

Guide 

Co-Lcarner/Co-investigator 


3 1 

2 1 
0 0 


3 3 
3 1 
1 1 


Student Roles 
Explorer* 

Cognitive apprentice 

Teacher 

Producer 


3 1 
0 0 
3 0 
3 0 


3 3 
1 

2 1 

3 3 


Totali for Design Sc Practice 


36 12 


64 4V 



High Performance Technology 
Indicators 


Program 
A Scores 


Program 
E Scores 




Dsgn Prtc 


Dsgn Prtc 


Access 

Connectivity 

Ubiquity 

Interactivity 

Design for equitable use 


1 1 
3 3 
3 3 
3 3 


3 2 
3 1 
3 3 
3 1 


OperabQity 

Interoperability 
Open architecture 
Transparency 


0 0 
0 0 
3 3 


3 3 
3 3 
3 1 


Resource Organization 

Distributed resources 
User contributions 
Collab. Projects/Design 


0 0 
0 0 
3 3 


3 2 
3 1 

3 2 


Engagement 

Opportunities for challenging 

tasks and experiences 
Opportunities to learn by doing 
Guided participation and 
intelligent tutoring 
Info, just in time and just enough 


2 1 

0 0 

3 3 

3 1 


3 2 

1 1 
3 1 

2 1 


Ease of Use 
Effective helps 
User friendliness 
User control 
Training and support 
Speed 


3 3 
3 3 
3 1 
3 1 
3 0 


3 1 
3 3 
3 2 

2 1 

3 3 


Functionality 

Use of multimedia technologies 

Use of generic tools 

Use of coEtext-apecific tools 

Programming and authoring skills 

Design and project skills 

Use of tools to make tools 


0 0 
3 1 
0 0 
0 0 
0 0 
0 0 


3 3 
3 2 

o o 1 

3 1 

3 2 
















42 30 


65 42 



y* ine 

* Indicators provided by Means, et a!., 1993 
*• All numbers represent hypothetical scores. 
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Legend for Table 7: Comparing Technology Program 



Des - Design. This column refers to features that are present in the design features as stated 
in formal descriptions of the program such as articles, profiles, promotional materials. 

Prtc = Practice. This column refers to features of the program that are typical in schools and other 
settings where the program is used. Sometimes there is a discrepancy between what the manual or description 
says and what the teachers actually do in the classroom. So this tool gives users the opportunity to evaluate 
the program as it was designed — its maximal potential— as well as the program as it is practiced. 



Scales for Design Scores for Learning and Technology 

0 = Not in place at this time/not applicable 

1 = Design definition in place but feature in program falls short of potential stated in the definition 

(Example: program has an encyclopedia for student exploration but it is very poor quality.) 

2 = Design definition in place and corresponds clearly to one or more features in the program 

(Example: program has an encyclopedia and it is functioning as described in literature but is not 
outstanding.) 

3 = Design definition in place and is a major appeal of the program 

(Example: Program has an encyclopedia and it is a major strength of the program.) 



Scales for Practice Scores for Learning and Technology 

0 = Not in place at this time/not applicable 

1 = Feature in place with no data to support 

2 = Feature clearly in place but only preliminary or limited data available 

3 = Strong empirical evidence that this feature of the program is in place and effective. 

Rationale for Hypothetical Data in Table 7 

Program A in Table 7 is an integrated learning system (ILS). Program B is a computer-driven approach 
involving integrated multimedia designed from the research from cognitive science. As with any program, both 
Program A and Program B have higher Design Scores than Practice Scores. 

The hypothetical ILS portrayed in Table 7 has strengths in its emphasis on heterogeneous groups and 
collaborative lemming, which is reflected in the Teacher and Student Roles as well as the Vision of Learning. 
Moreover, it is highly interactive and has a capacity for assessment that is aligned with curriculum and 
instruction and is ongoing. 

The hypothetical Program B in Table 7 v/as designed to link students to the Internet so that they could engage in 
co-investigations with students from other schools and with practicing professionals. So this Program has a lot 
of Access, Operability, and Resource capabilities that Program A does not have, and Program B has higher 
Learning scores for Design and Practice. Thus, for example, Program B has high scores for all the Access 
items: Connectivity, Ubiquity, and Interactivity. Similarly, such a program would have high scones for the 
Operability items: Interoperability, Open Architecture, and Transparency. 
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TABLE 7a: COMPARING TECHNOLOGY PROGRAMS 
FOR ENGAGED LEARNING AND HIGH PERFORMANCE TECHNOLOGY 



Engaged Learning 
Indicators 


Program 
A Scores 


Program 
B Scores 




Dsgn Prtc 


Dsgn Prtc 


Vision of Learning 
Responsible for learning 
Strategic 

Energized by learning 
Collaborative 






Tasks 

Authentic 4 
Challenging 4 

Integrative/Intertfisc iplinary * 






Assessment 

Pe rformanc e-b i se d * 

Generative 

Seamless and ongoing 
Equitable standarda 






Instruction/Model 

Interactive* 

Generative 






Learning Context 
Collaborative* 
Knowledge building 
Empathetic 
Multiple perspectives 






Grouping 
Heterogeneous* 
Equitable 
Flexible 






Teacher Roles 

Facilitator* 

Guide 

Co-Lea rner/Co-inveitigator 






Student Roles 
Explorer* 

Cognitive apprentice 

Teacher 

Producer 






Totals for Deaign & Practice 







High Performance Technology 
Indicators 


Program 
A Scores 


Program 
B Scores 




Dsgn Prtc 


Dsgn Prtc 


Access 

Connectivity 

Ubiquity 

Interactivity 

Design for equitable use 






0 Durability 

Interoperability 
Open architecture 
Transparency 






Resource Organization 
Distributed resources 
User contributions 
Collsb. Projects/Design 






Engagement 

Opportunities for challenging 

tasks and experiences 
Opportunities to learn by doing 
Guided participation and 

intelligent tutoring 
Info, just in time and just enough 






Ease of Use 
Effective helps 
User friendliness 
User control 
Training and support 
Speed 






Functionality 

Use of multimedia technologies 
Use of basic tools 
Generic 

Context specific 
Develops programming and 
authoring 

Develops design Sl project akilla 
Use of tools to make tools 




— 





















Copyright c 1994 The North" Central Regional Educational Laboratory 
* Indicator! provided by Mean*, et al. ( 1993 
All numbera reprtaent hypothetical acorea. 
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Legend for Table 7a: Comparing Technology Programs 

Desn = Design. This column refers to features that are present in the design features as stated 
in formal descriptions of the program such as articles, profiles, promotional materials. 

Prtc = Practice. This column refers to features of the program that are typical in schools and other 
settings where the program is used. Sometimes there is a discrepancy between what the manual or description 
says and what the teachers actually do in the classroom. So this tool gives users the opportunity to evaluate 
the program as is was designed — its maximal potential— as well as the program as it is practiced. 



Scales c or Design Scores for Learning and Technology 

0 = Not in place at this time/not applicable 

1 = Design definition in place but feature in program falls short of potential stated in the definition 

(Example: program has an encyclopedia for student exploration but it is very poor quality.) 

2 = Design definition in place and corresponds clearly to one or more features in the program 

(Example: program has an encyclopedia and it is functioning as described in literature but is not 
outstanding.) 

3 = Design definition in place and is a major appeal ok the program. 

(Example: Program has an encyclopedia and it is a major strength of the program.) 



Scales for Practice Scores for Learning and Technology 

0 = Not in place at this time/not applicable 

1 » Feature in place with no data to support 

2 = Feature clearly in place but only preliminary or limited data available 

3 = Strong empirical evidence that this feature of the program is in place and effective. 
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TABLE 8 

COMPARING TECHNOLOGY PROGRAMS 
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Legend for Table 8 



Calculations for Program A 

Using Table 7, column totals, calculate the Learning Grand Total Score: 



Program A Learning Design Column Total 36 
Program A Learning Practice Column Total + 12 



Learning Grand Total 48 
Place a circle on the horizontal axis at about 48. 

Using Table 7 again, calculate the Technology Grand Total Score: 
Program A Technology Design Column Total 42 
Program A Technology Practice Column Total + 30 



Technology Grand Total 72 
Place an X on the vertical axis at about 72. 



For the full profile of Program A, draw a solid line upward from the circle and sideways from the X to the 
intersection. 

Calculations for Program B 

Using Table 7, column totals, calculate the Learning Grand Total Score: 



Program B Learning Design Column Total 64 
Program B Learning Practice Column Total -f 49 

Learning Grand Total 1 13 

Place a circl e on the horizontal axis at about 113. 

Using Table 7 again, calculate the Technology Grand Total Score: 
Program B Technology Design Column Total 65 
Program B Technology Practice Column Total +41 

Technology Grand Total 106 

Place an X on the vertical axis at about 106. 



For the full profile of Program B, draw a dashed line upward from the circle and sideways from the X to the 
intersection. 
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COMPARING TECHNOLOGY PROGRAMS 
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Legend for Table 8a 



Calculations for Program A 

Using Table 7a, column totals, calculate the Learning Grand Total Score: 
Program A Learning Design Column Total 
Program A Learning Practice Column Total + 



Learning Grand Total 

Place a circle on the horizontal axis for the Learning Grand Total for Program A. 

Using Table 7a again, calculate the Technology Grand Total Score: 
Program A Technology Design Column Total 
Program A Technology Practice Column Total + 



Technology Grand Total 

Place an X on the vertical axis for the Technology Grand Total for Program A. 

For the full profile of Program A, draw a solid line upward from the circle and sideways from the X to the 
intersection. 



Calculations for Program B 

Using Table 7a, column totals, calculate the Learning Grand Total Score: 
Program B Learning Design Column Total 
Program B Learning Practice Column Total + 

Learning Grand Total 

Place a circle on the horizontal axis for the Learning Grand Total for Program B. 



Using Table 7a again, calculate the Technology Grand Total Score: 
Program B Technology Design Column Total 
Program B Technology Practice Column Total + 

Technology Grand Total 

Place an X on the vertical axis for the Technology Grand Total for Program B. 

For the full profile of Program B, draw a dashed line upward from the circle and sideways from the X to 
intersection. 
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APPENDICES D-l AND D-2: COST CHECKLISTS 
D-l: ASSESSING ALL THE COSTS 



Instructions: This appendix is provided simply as a tool for the school or district to raise questions for school 
management to consider as they identify and compare various technologies and programs. Reproduce it and fill 
it out for various vendors. 



Actual Costs 

Single/Multiple Users 

License 

One-Time Purchase 

Yearly Support Fee 

900 Number 

Upgrade costs 

Installation 

Hardware Required 

Supplies (paper, ribbons, etc.) 

Telecommunications 

Lines 

Modems 

Connect charges 

Long distance charges 

Training 

Students 

Teachers 

Technicians 

Other 

Security 

Building 

Information 

System operations 

Data 

_ Insurance increases 

Processing 



Vendor Name 



USE THIS SPACE FOR NOTES 
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Disc size 

Alternative storage 

Alternative power 

CD-ROM 

Tapes 

Future expansion 

Connectivity 

Backups 

Expansion boards 

Video Production 

Cameras 

Monitors 

Video editing equipment 

Laserdisc player 

Jukebox 

CD-ROM player 

Headphones and/or speakers 

Audioboard 

Software 

Building Requirements 

Heating and cooling 

Cabling plant 

'Clean power" (uninterrupted power) 
Inter-Campus Communications 

Special Needs 

Delivers to special populations (e.g., deaf/blind) 

Delivers to special locations (e.g., rural/hospitals) 

Disclosure about Costs 

___Purchase costs 

Software 

Hardware 

Installation time 

Professional development 

Maintenance Costs 

Expenses 

Time 

Equipment 

M aterials 

Warranties and Guarantees 
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Upgrading Costs 

Possible reentry of data 

Material costs 

Hardware and software 

Legacy hard, soft, data 

Growth Capabilities, Extendibility, Upgrades 

Hardware 

Software 

Ability to import existing data into upgraded version 

Ability to automate or charge software functions to meet unique needs 

Ability to do macros, customized menus, user-selected defaults, etc. 

Ability to add powerful telecommunications such as 56K lines, Integrated Services Digital Network 

(ISDN), Tl and T3 lines 

Economy of Use 

Site licenses 

Large data sets easily downloaded during off hours 

Telecommunications software allows composition and reading on site rather than online 

Effective dissemination regarding software and free resources 

Support 

Quick, up to speed 
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APPENDIX D-2: VENDOR QUALITY 



Instructions: This appendix is provided simply as a tool for the school or district to raise questions for school 
management to consider as they identify and compare various technologies and programs. Reproduce it and fill 
it out for various vendors. 



Stability of Software Vendor 

Size of vendor 

Years in business 

Reputation 

Product Line 

One-product company 

Many products 

Research-based 

Focus on Learning and student control (vs. teaching) 

Support Staff 

Number available 

Experience 

K nowledge about research on learning 
Willingness to go beyond duty 

Willingness to provide evidence of successful learning 

Supporting Evidence 

Studies and literature focused on learning 

Studies conducted by company 

Studies conducted by external evaluator 

Diversity of data sources 

Citations/Feviews in the research literature 

Alignment with Legacy Software and Equipment 



Vendor Name 



USE THIS SPACE FOR NOTES 
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Instructions: This appendix is provided simply as a tool for the school or district to raise questions for school 
management to consider as they identify and compare various technologies and programs. 



Vendor B 



USE THIS SPACE FOR NOTES 



Stability of Software Vendor 

Size of vendor 

Years in business 

Reputation 

Product Line 

One-product company 

Many products 

Research-based 

Focus on Learning and student control (vs. teaching) 

Support Staff 

Number available 

Experience 

Knowledge about research on learning 

W illingness to go beyond duty 
Willingness to provide evidence of successful learning 

Supporting Evidence 

Studies and literature focused on learning 

Studies conducted by company 

Studies conducted by external evaluator 

Diversity of data sources 

Citations/Reviews in the research literature 

Alignment with Legacy Software and Equipment 
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